U
e5d @ s d Z ddlZddlZddlZddlZddlZddlZz ddlZW n ek
rX dZ Y nX dZ ddl
m Z ddlm
Z
ddd d
d d d
dgZdZG dd deZG dd d eZG dd
d
eZG dd d eZG dd d eZG dd
d
eZdZdZddddd d d d d d!d"d#h Zd$d%d&d'd(d)d*gZd)d*d+Zd,Z e d-d.d/d0d1gZ e d2d3d4d5gZ d6d Z d7d8 Z dmd9d:Z!dnd;d<Z"dod=d>Z#e rd?d@ Z$G dAdB dBZ%G dCd de%Z&e rG dDdE dEe%Z'e(dE e)dFkrddl*Z*e*j+dGdHZ,e,j-dIdJdKdLdM e,j-dNdOdPdQdM e,j-dRdSdTe.dUeef dV e,j-dWdXdYe.dZdV e,j-d[d\d]dd^d_ e,/ Z0e0j1Z1e0jse1dTkr|eZ1e&e0j2e1d`Z3n e1dTkreZ1e'e0j2e1d`Z3e34 Z5dae5kre36 e37e0j7\Z8Z9Z:Z;Ze?e.e;e0j@ dh Z:e3Ae:e;\Z8ZBeBD ]Z\ZCZDe eDd% Edidhd ZFe eDd$ ZGe.eDd* ZHe=djIeCe>eFdke>eGdleH q e3J dS )pa An NNTP client class based on:
- RFC 977: Network News Transfer Protocol
- RFC 2980: Common NNTP Extensions
- RFC 3977: Network News Transfer Protocol (version 2)
Example:
>>> from nntplib import NNTP
>>> s = NNTP('news')
>>> resp, count, first, last, name = s.group('comp.lang.python')
>>> print('Group', name, 'has', count, 'articles, range', first, 'to', last)
Group comp.lang.python has 51 articles, range 5770 to 5821
>>> resp, subs = s.xhdr('subject', '{0}-{1}'.format(first, last))
>>> resp = s.quit()
>>>
Here 'resp' is the server response line.
Error responses are turned into exceptions.
To post an article from a file:
>>> f = open(filename, 'rb') # file containing article, including header
>>> resp = s.post(f)
>>>
For descriptions of all methods, read the comments in the code below.
Note that all arguments and return values representing article numbers
are strings, not numbers, since they are rarely used for calculations.
NFT)
decode_header)_GLOBAL_DEFAULT_TIMEOUTNNTP NNTPErrorNNTPReplyErrorNNTPTemporaryErrorNNTPPermanentErrorNNTPProtocolError
NNTPDataErrorr i c @ s e Zd ZdZdd ZdS )r z%Base class for all nntplib exceptionsc G s> t j| f| z|d | _W n tk
r8 d| _Y nX d S )Nr zNo response given) Exception__init__response
IndexError)selfargs r /usr/lib64/python3.8/nntplib.pyr c s
zNNTPError.__init__N)__name__
__module__ __qualname____doc__r r r r r r a s c @ s e Zd ZdZdS )r zUnexpected [123]xx replyNr r r r r r r r r j s c @ s e Zd ZdZdS )r z
4xx errorsNr r r r r r n s c @ s e Zd ZdZdS )r z
5xx errorsNr r r r r r r s c @ s e Zd ZdZdS )r z"Response does not begin with [1-5]Nr r r r r r v s c @ s e Zd ZdZdS )r
zError in response dataNr r r r r r
z s w i3 Z100Z101211215Z220Z221Z222Z224Z225Z230Z231Z282subjectfromdatez
message-idZ
referencesz:bytesz:lines)byteslines
GroupInfogrouplastfirstflag ArticleInfoZnumber
message_idr c C sJ g }t | D ]2\}}t|tr4|||p,d q || q d|S )zvTakes a unicode string representing a munged header value
and decodes it as a (possibly non-ASCII) readable value.ascii )_email_decode_header
isinstancer appenddecodejoin)Z
header_strpartsvencr r r r s
c C s g }| D ]`}|d dkr:|dd d\}}}d| }n| d\}}}| }t||}|| qt}t|t|k rtd|dt| |krtd|S )zParse a list of string representing the response to LIST OVERVIEW.FMT
and return a list of header/metadata names.
Raises NNTPDataError if the response is not compliant
(cf. RFC 3977, section 8.4).r : Nz$LIST OVERVIEW.FMT response too shortz*LIST OVERVIEW.FMT redefines default fields) partitionlower_OVERVIEW_FMT_ALTERNATIVESgetr, _DEFAULT_OVERVIEW_FMTlenr
)r fmtlinename_suffixdefaultsr r r _parse_overview_fmt s
r@ c C s t t}g }| D ]}i }|d^}}t|}t|D ]\} }
| t |krLq6|| } | d} | |kr| s| d }
|
r|
dt |
|
krtd|
r|
t |
d nd}
|
||| < q6|||f q|S )zZParse the response to an OVER or XOVER command according to the
overview format `fmt`. r2 z: Nz?OVER/XOVER response doesn't include names of additional headers) r9 r8 splitint enumerate
startswithr5 r
r, )r r: Zdata_process_funcZ
n_defaultsZoverviewr; ZfieldsZarticle_numbertokensitokenZ
field_nameZ is_metadatahr r r _parse_overview s&
rJ c C s |dkr | dd }| dd } t |dd }t |dd }t |dd }t | dd }t | dd }t | dd }|dk r|d7 }n|d k r|d
7 }t||||||S ) zParse a pair of (date, time) strings, and return a datetime object.
If only the date is given, it is assumed to be date and time
concatenated together (e.g. response to the DATE command).
Ni F i d il )rC datetime)date_strtime_strZhoursZminutesZsecondsyearZmonthZdayr r r _parse_datetime s
rU c C sP t | tjsd}n
d| }| j}|r<|d }d|| }n d|| }||fS )aP Format a date or datetime object as a pair of (date, time) strings
in the format required by the NEWNEWS and NEWGROUPS commands. If a
date object is passed, the time is assumed to be midnight (00h00).
The returned representation depends on the legacy flag:
* if legacy is False (the default):
date has the YYYYMMDD format and time the HHMMSS format
* if legacy is True:
date has the YYMMDD format and time the HHMMSS format.
RFC 3977 compliant servers should understand both formats; therefore,
legacy is only needed when talking to old servers.
Z000000z({0.hour:02d}{0.minute:02d}{0.second:02d}rP z {0:02d}{1.month:02d}{1.day:02d}z {0:04d}{1.month:02d}{1.day:02d})r+ rQ formatrT )ZdtZlegacyrS yrR r r r _unparse_datetime s
rX c C s |dkrt }|j| |dS )zWrap a socket in SSL/TLS. Arguments:
- sock: Socket to wrap
- context: SSL context to use for the encrypted connection
Returns:
- sock: New, encrypted socket.
N)Zserver_hostname)sslZ_create_stdlib_contextZ wrap_socket)sockcontextZhostnamer r r _encrypt_on s r\ c @ s e Zd ZdZdZdefddZdd Zdd Zd
d Z d d
Z
dd Z e Z dd Z
dd ZdhddZdd ZdiddZdd Zdjd d Zdkd d Zd!d" Zd#d$ Zd%d& Zdd'd(d)Zdd'd*d+Zdldd'd,d-Zd.d/ Zd0d1 Z d2d3 Z d4d5 Z dd'd6d7Z d8d9 Z d:d; Z!dmdd? Z#d@dA Z$dndBdCZ%dodd'dDdEZ&dpdd'dFdGZ'dqdd'dHdIZ(dJdK Z)dd'dLdMZ*dd'dNdOZ+dd'dPdQZ,dd'dRdSZ-dTdU Z.dVdW Z/dXdY Z0dZd[ Z1d\d] Z2d^d_ Z3d`da Z4drdbdcZ5ddde Z6e7rdsdfdgZ8dS )t _NNTPBasezutf-8surrogateescapeNc C sj || _ || _d| _| | _d| _| d| _|rZd| jkrZ| | jsZd| _| d| _ d| _
dS )aS Initialize an instance. Arguments:
- file: file-like object (open for read/write in binary mode)
- host: hostname of the server
- readermode: if true, send 'mode reader' command after
connecting.
- timeout: timeout (in seconds) used for socket connections
readermode is sometimes necessary if you are connecting to an
NNTP server on the local machine and intend to call
reader-specific commands, such as `group'. If you get
unexpected NNTPPermanentErrors, you might need to set
readermode.
r NFREADER) hostfile debugging_getrespwelcome_capsgetcapabilitiesreadermode_afterauth_setreadermodetls_on
authenticated)r ra r`
readermodetimeoutr r r r 9 s
z_NNTPBase.__init__c C s | S Nr r r r r __enter__h s z_NNTPBase.__enter__c sR fdd}| rNz*z W n ttfk
r8 Y nX W 5 | rL X d S )Nc s
t dS )Nra )hasattrr rn r r l z$_NNTPBase.__exit__..)_closequitOSErrorEOFError)r r Z is_connectedr rn r __exit__k s
z_NNTPBase.__exit__c C s | j rtdt| j | jS )zGet the welcome message from the server
(this is read and squirreled away by __init__()).
If the response code is 200, posting is allowed;
if it 201, posting is not allowed.z *welcome*)rb printreprrd rn r r r
getwelcomev s z_NNTPBase.getwelcomec C s | j dkrd| _d| _z| \}}W n ttfk
rD i | _ Y n s z,_NNTPBase._longcmdstring..r )r r; ra r listr rn r _longcmdstring s
z_NNTPBase._longcmdstringc C sd z| j W S tk
r Y nX z| d\}}W n tk
rP tdd }Y n
X t|}|| _ |S )zqInternal: get the overview format. Queries the server if not
already done, else returns the cached value.zLIST OVERVIEW.FMTN)Z_cachedoverviewfmtAttributeErrorr r r8 r@ )r r r r: r r r _getoverviewfmt s z_NNTPBase._getoverviewfmtc C s dd |D S )Nc S s g | ]}t | qS r )r! rB r r r r r & s z(_NNTPBase._grouplist..r )r r r r r
_grouplist$ s z_NNTPBase._grouplistc C s8 i }| d\}}|D ]}| ^}}|||< q||fS )a! Process a CAPABILITIES command. Not supported by all servers.
Return:
- resp: server response if successful
- caps: a dictionary mapping capability names to lists of tokens
(for example {'VERSION': ['2'], 'OVER': [], LIST: ['ACTIVE', 'HEADERS'] })
Z CAPABILITIES)r rB )r r r r r; r< rF r r r r} ( s
z_NNTPBase.capabilities)ra c C sb t |tjtjfs$td|jjt|| jdk \}}d||}| ||\}}||
|fS )zProcess a NEWGROUPS command. Arguments:
- date: a date or datetime object
Return:
- resp: server response if successful
- list: list of newsgroup names
Athe date parameter must be a date or datetime object, not '{:40}'rK zNEWGROUPS {0} {1}) r+ rQ r TypeErrorrV __class__r rX r| r r )r r ra rR rS cmdr r r r r newgroups6 s z_NNTPBase.newgroupsc C sR t |tjtjfs$td|jjt|| jdk \}}d|||}| ||S )zProcess a NEWNEWS command. Arguments:
- group: group name or '*'
- date: a date or datetime object
Return:
- resp: server response if successful
- list: list of message ids
r rK zNEWNEWS {0} {1} {2})
r+ rQ r r rV r r rX r| r )r r" r ra rR rS r r r r newnewsF s z_NNTPBase.newnewsc C s4 |dk rd| }nd}| ||\}}|| |fS )a@ Process a LIST or LIST ACTIVE command. Arguments:
- group_pattern: a pattern indicating which groups to query
- file: Filename string or file object to store the result in
Returns:
- resp: server response if successful
- list: list of (group, last, first, flag) (strings)
Nz LIST ACTIVE ZLIST)r r )r
group_patternra commandr r r r r r V s
z_NNTPBase.listc C s t d}| d| \}}|ds8| d| \}}i }|D ]:}|| }|r@|dd\} }
|sr|
S |
|| < q@|r||fS dS d S )Nz ^(?P[^ ]+)[ ]+(.*)$zLIST NEWSGROUPS r XGTITLE r3 rK r) )recompiler rE searchstripr" ) r r Z
return_allline_patr r groupsraw_linematchr< Zdescr r r _getdescriptionse s
z_NNTPBase._getdescriptionsc C s | |dS )a Get a description for a single group. If more than one
group matches ('group' is a pattern), return the first. If no
group matches, return an empty string.
This elides the response code from the server, since it can
only be '215' or '285' (for xgtitle) anyway. If the response
code is needed, use the 'descriptions' method.
NOTE: This neither checks for a wildcard in 'group' nor does
it check whether the group actually exists.Fr )r r" r r r description| s z_NNTPBase.descriptionc C s | |dS )z'Get descriptions for a range of groups.Tr )r r r r r descriptions s z_NNTPBase.descriptionsc C s | d| }|ds t|| }d } }}t|}|dkr|d }|dkr|d }|dkr|d }|dkr|d }|t|t|t||fS )a Process a GROUP command. Argument:
- group: the group name
Returns:
- resp: server response if successful
- count: number of articles
- first: first article number
- last: last article number
- name: the group name
zGROUP r r r3 rK r rL )r rE r rB r9 r5 rC )r r< r wordscountr$ r# nr r r r" s
z_NNTPBase.groupc C s | d|S )a Process a HELP command. Argument:
- file: Filename string or file object to store the result in
Returns:
- resp: server response if successful
- list: list of strings returned by the server in response to the
HELP command
ZHELP)r )r ra r r r help s z_NNTPBase.helpc C s8 | dst|| }t|d }|d }|||fS )z_Internal: parse the response line of a STAT, NEXT, LAST,
ARTICLE, HEAD or BODY command.Z22r3 rK )rE r rB rC )r r r art_numr' r r r
_statparse s
z_NNTPBase._statparsec C s | |}| |S )z/Internal: process a STAT, NEXT or LAST command.)r r )r r; r r r r _statcmd s
z_NNTPBase._statcmdc C s" |r| d|S | dS dS )a( Process a STAT command. Argument:
- message_spec: article number or message id (if not specified,
the current article is selected)
Returns:
- resp: server response if successful
- art_num: the article number
- message_id: the message id
zSTAT {0}ZSTATN)r rV )r message_specr r r stat s z_NNTPBase.statc C s
| dS )z;Process a NEXT command. No arguments. Return as for STAT.ZNEXTr rn r r r next s z_NNTPBase.nextc C s
| dS )z;Process a LAST command. No arguments. Return as for STAT.ZLASTr rn r r r r# s z_NNTPBase.lastc C s0 | ||\}}| |\}}}|t|||fS )z2Internal: process a HEAD, BODY or ARTICLE command.)r r r& )r r; ra r r r r' r r r _artcmd s z_NNTPBase._artcmdc C s$ |dk rd |}nd}| ||S )a0 Process a HEAD command. Argument:
- message_spec: article number or message id
- file: filename string or file object to store the headers in
Returns:
- resp: server response if successful
- ArticleInfo: (article number, message id, list of header lines)
NzHEAD {0}ZHEADrV r r r ra r r r r head s z_NNTPBase.headc C s$ |dk rd |}nd}| ||S )a+ Process a BODY command. Argument:
- message_spec: article number or message id
- file: filename string or file object to store the body in
Returns:
- resp: server response if successful
- ArticleInfo: (article number, message id, list of body lines)
NzBODY {0}ZBODYr r r r r body s z_NNTPBase.bodyc C s$ |dk rd |}nd}| ||S )a5 Process an ARTICLE command. Argument:
- message_spec: article number or message id
- file: filename string or file object to store the article in
Returns:
- resp: server response if successful
- ArticleInfo: (article number, message id, list of article lines)
Nz ARTICLE {0}ZARTICLEr r r r r article s z_NNTPBase.articlec C s
| dS )zYProcess a SLAVE command. Returns:
- resp: server response if successful
ZSLAVE)r rn r r r slave s z_NNTPBase.slavec sD t d | d|||\}} fdd|fdd|D fS )ai Process an XHDR command (optional server extension). Arguments:
- hdr: the header type (e.g. 'subject')
- str: an article nr, a message id, or a range nr1-nr2
- file: Filename string or file object to store the result in
Returns:
- resp: server response if successful
- list: list of (nr, value) strings
z^([0-9]+) ?(.*)
?z XHDR {0} {1}c s | }|r|ddS | S )Nr3 rK )r r" )r; m)patr r
remove_number s
z%_NNTPBase.xhdr..remove_numberc s g | ] } |qS r r r )r r r r s z"_NNTPBase.xhdr..)r r r rV )r Zhdrr ra r r r )r r r xhdr s
z_NNTPBase.xhdrc C s. | d|||\}}| }|t||fS )aF Process an XOVER command (optional server extension) Arguments:
- start: start of range
- end: end of range
- file: Filename string or file object to store the result in
Returns:
- resp: server response if successful
- list: list of dicts containing the response fields
z
XOVER {0}-{1})r rV r rJ )r startendra r r r: r r r xover s
z_NNTPBase.xoverc C sx d| j krdnd}t|ttfr>|\}}|d||p6d7 }n|dk rR|d | }| ||\}}| }|t||fS )a Process an OVER command. If the command isn't supported, fall
back to XOVER. Arguments:
- message_spec:
- either a message id, indicating the article to fetch
information about
- or a (start, end) tuple, indicating a range of article numbers;
if end is None, information up to the newest message will be
retrieved
- or None, indicating the current article number must be used
- file: Filename string or file object to store the result in
Returns:
- resp: server response if successful
- list: list of dicts containing the response fields
NOTE: the "message id" form isn't supported by XOVER
ZOVERZXOVERz {0}-{1}r) Nr{ )re r+ tupler rV r r rJ ) r r ra r r r r r r: r r r over* s z_NNTPBase.overc C sf t dtd td}| d| |\}}g }|D ](}|| }|r4|| dd q4||fS )zProcess an XGTITLE command (optional server extension) Arguments:
- group: group name wildcard (i.e. news.*)
Returns:
- resp: server response if successful
- list: list of (name,title) stringszFThe XGTITLE extension is not actively used, use descriptions() insteadrK z^([^ ]+)[ ]+(.*)$r r3 )
warningswarnDeprecationWarningr r r r r r, r" ) r r" ra r r Z raw_linesr r r r r r xgtitleE s
z_NNTPBase.xgtitlec C sl t dtd | d|}|ds0t|z| \}}W n tk
r^ t|dY n
X ||fS dS )zProcess an XPATH command (optional server extension) Arguments:
- id: Message id of article
Returns:
resp: server response if successful
path: directory path to article
z(The XPATH extension is not actively usedrK z XPATH {0}Z223N) r r r r rV rE r rB
ValueError)r idr Zresp_numpathr r r xpathW s
z_NNTPBase.xpathc C sb | d}|ds t|| }t|dkr8t||d }t|dkrTt||t|dfS )zProcess the DATE command.
Returns:
- resp: server response if successful
- date: datetime object
ZDATEZ111rK r3 N)r rE r rB r9 r
rU )r r elemr r r r r k s
z_NNTPBase.datec C s | |}|ds t|t|ttfr2| }|D ]:}|tsR| dt }|drdd| }| j
| q6| j
d | j
|
S )N3r r s .
)r rE r r+ r bytearray
splitlinesendswithr rstripra r r rc )r r fr r; r r r _post| s
z_NNTPBase._postc C s | d|S )zProcess a POST command. Arguments:
- data: bytes object, iterable or file containing the article
Returns:
- resp: server response if successfulZPOST)r )r datar r r post s z_NNTPBase.postc C s | d||S )a Process an IHAVE command. Arguments:
- message_id: message-id of the article
- data: file containing the article
Returns:
- resp: server response if successful
Note that if the server refuses the article an exception is raised.z IHAVE {0})r rV )r r' r r r r ihave s z_NNTPBase.ihavec C s | j | ` d S rm )ra r rn r r r rs s
z_NNTPBase._closec C s z| d}W 5 | X |S )zdProcess a QUIT command and close the socket. Returns:
- resp: server response if successfulZQUIT)rs r )r r r r r rt s
z_NNTPBase.quitc C s | j rtd|s |s tdz<|rX|sXdd l}| }|| j}|rX|d }|d }W n tk
rn Y nX |sxd S | d| }|dr|st|n | d| }|dst |d | _
| | j rd | j
kr|
d | _
| d S )
NzAlready logged in.z7At least one of `user` and `usenetrc` must be specifiedr rK zauthinfo user Z381zauthinfo pass Z281r_ )rj r netrcZauthenticatorsr` ru r rE r r re rf rg rh )r userpasswordusenetrcr Z credentialsZauthr r r r login s>
z_NNTPBase.loginc
C s` z| d| _W nJ tk
r$ Y n8 tk
rZ } z|jdrHd| _n W 5 d }~X Y nX d S )Nz mode readerZ480T)r rd r r r
rE rg )r er r r rh s z_NNTPBase._setreadermodec C s| | j rtd| jr td| d}|drp| j t| j|| j | _| j
d| _d| _ d| _ | nt
ddS ) zzProcess a STARTTLS command. Arguments:
- context: SSL context to use for the encrypted connection
zTLS is already enabled.z+TLS cannot be started after authentication.STARTTLSZ382rwbTNzTLS failed to start.)ri r rj r rE ra r r\ rZ r` makefilere rf r )r r[ r r r r starttls s
z_NNTPBase.starttls)T)N)N)N)N)N)N)N)N)N)NNT)N)9r r r r r r r ro rw rz rf r debugr r r rc r r r r r r r} r r r r r r r" r r r r r r# r r r r r r r r r r r r r r rs rt r rh _have_sslr r r r r r] ) sn
/
.
)
r] c @ s* e Zd ZeddddefddZdd ZdS )r NFc C s || _ || _td| || t||f|| _d}z8| jd}t | |||| |sZ|rh|
||| W n$ |r~| | j Y nX dS )a, Initialize an instance. Arguments:
- host: hostname to connect to
- port: port to connect to (default the standard NNTP port)
- user: username to authenticate with
- password: password to use with username
- readermode: if true, send 'mode reader' command after
connecting.
- usenetrc: allow loading username and password from ~/.netrc file
if not specified explicitly
- timeout: timeout (in seconds) used for socket connections
readermode is sometimes necessary if you are connecting to an
NNTP server on the local machine and intend to call
reader-specific commands, such as `group'. If you get
unexpected NNTPPermanentErrors, you might need to set
readermode.
nntplib.connectNr ) r` portr r socketcreate_connectionrZ r r] r r r ) r r` r r r rk r rl ra r r r r s$
z
NNTP.__init__c C s zt| W 5 | j X d S rm rZ r r] rs rn r r r rs $ s z NNTP._close)r r r NNTP_PORTr r rs r r r r r s
%c @ s, e Zd ZedddddefddZdd ZdS )NNTP_SSLNFc
C s t d| || t||f|| _d} zJt| j||| _| jd} tj| | |||d |s`|rn| ||| W n$ | r|
| j
Y nX dS )zThis works identically to NNTP.__init__, except for the change
in default port and the `ssl_context` argument for SSL connections.
r Nr )rk rl ) r r r r rZ r\ r r] r r r )
r r` r r r Z ssl_contextrk r rl ra r r r r . s"
zNNTP_SSL.__init__c C s zt| W 5 | j X d S rm r rn r r r rs E s zNNTP_SSL._close)r r r
NNTP_SSL_PORTr r rs r r r r r , s
r __main__zJ nntplib built-in demo - display the latest articles in a newsgroup)r z-gz--groupzgmane.comp.python.generalz3group to fetch messages from (default: %(default)s))defaultr z-sz--serverz
news.gmane.ioz+NNTP server hostname (default: %(default)s)z-pz--portr z#NNTP port number (default: %s / %s))r typer z-nz
--nb-articles
z2number of articles to fetch (default: %(default)s)z-Sz--ssl
store_truezuse NNTP over SSL)actionr r )r` r r ZGroupZhaszarticles, rangetoc C s$ t | |kr | d |d d } | S )NrL z...)r9 )sZlimr r r cutp s r
r3 s A
Y.