Wj f d Z ddlZddlZddlZddlZddlZddlZddlZddl Z ddl
Z ddl m
Z
g dZdZdZdZdZd Zd
Z e j- ej. j0 ej. j0 j3 D ci c] } | | j4 c} Zd Z d Z d
Z ej> d j@ Z! ej> d jD Z# ej> d Z$ ej> d Z%h dZ&d?dZ'de(de(fdZ) G d dejT jV Z,d Z-e,fdZ.e,fdZ/ G d d ej` Z1d Z2 G d d Z3 ddl4Z4 G d! d"e3 Z5ejm d" G d# d$e8 Z9 G d% d&e9 Z: G d' d(e9 Z; G d) d*e9 Z< G d+ d,e9 Z= G d- d.e9 Z> G d/ d0e9 Z? G d1 d2e9 Z@ G d3 d4e@ ZA G d5 d6e@ ZB G d7 d8e@ ZC G d9 d:e9 ZD G d; deFeD ZGe9ZHyc c} w # e7$ r Y w xY w)@a
HTTP/1.1 client library
HTTPConnection goes through a number of "states", which define when a client
may legally make another request or fetch the response for a particular
request. This diagram details these state transitions:
(null)
|
| HTTPConnection()
v
Idle
|
| putrequest()
v
Request-started
|
| ( putheader() )* endheaders()
v
Request-sent
|\_____________________________
| | getresponse() raises
| response = getresponse() | ConnectionError
v v
Unread-response Idle
[Response-headers-read]
|\____________________
| |
| response.read() | putrequest()
v v
Idle Req-started-unread-response
______/|
/ |
response.read() | | ( putheader() )* endheaders()
v v
Request-started Req-sent-unread-response
|
| response.read()
v
Request-sent
This diagram presents the following rules:
-- a second request may not be started until {response-headers-read}
-- a response [object] cannot be retrieved until {request-sent}
-- there is no differentiation between an unread response body and a
partially read response body
Note: this enforcement is applied by the HTTPConnection class. The
HTTPResponse class does not enforce this state machine, which
implies sophisticated clients may accelerate the request/response
pipeline. Caution should be taken, though: accelerating the states
beyond the above pattern may imply knowledge of the server's
connection-close behavior for certain requests. For example, it
is impossible to tell whether the server will close the connection
UNTIL the response headers have been read; this means that further
requests cannot be placed into the pipeline until it is known that
the server will NOT be closing the connection.
Logical State __state __response
------------- ------- ----------
Idle _CS_IDLE None
Request-started _CS_REQ_STARTED None
Request-sent _CS_REQ_SENT None
Unread-response _CS_IDLE
Req-started-unread-response _CS_REQ_STARTED
Req-sent-unread-response _CS_REQ_SENT
N)urlsplit) HTTPResponseHTTPConnection
HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead
InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady
BadStatusLine LineTooLongRemoteDisconnectederror responsesP i UNKNOWNIdlezRequest-startedz Request-senti d i s [^:\s][^:\r\n]*s \n(?![ \t])|\r(?![ \t\n])z[ - ]z[ - ]> PUTPOSTPATCHc
| j d S # t $ rl}t |j |j |j |j
|j
d| |j |j
dd|d dd}~ww xY w)z bytes "should" be present for
reading. If the bytes are truly not available (due to EOF), then the
IncompleteRead exception can be used to detect the problem.
r rw ) min_MIN_READ_BUF_SIZErR r r= r ioBytesIOseekwritetellgetvalue)rA r cursizer& deltas r) r zHTTPResponse._safe_read s c-.ww||G$ t9 K t9w sSY7 7zz$ !Q w /E JJtww||E* +yy{c!}}& u Gyy{W$$T]]_cDIIK6GHH r6 c t | }| j j | }||k rt t |d| ||z
|S )z2Same as _safe_read, but for reading into a buffer.N)r= rR r r r )rA r r rB s r) r zHTTPResponse._safe_readinto sF !f GG Q s7 q!u s1u5 5r6 c | j | j dk( ry| j r| j | S | j |dk s|| j kD r | j }| j j | }|s|r| j
|S | j :| xj t | z c_ | j s| j
|S )zvRead with at most one underlying system call. If at least one
byte is buffered, return that instead.
r r6 r )rR rh rn _read1_chunkedrp read1r~ r= )rA rB results r) r zHTTPResponse.read1 s 77?dllf 4 <<&&q) ) ;; "AT[[ A q!!
[[
$ KK3v; &K;; "
r6 c | j | j dk( ry| j r| j | S | j j | S )Nr r6 )rR rh rn
_peek_chunkedpeek)rA rB s r) r zHTTPResponse.peek sE 77?dllf 4 <<%%a( (ww||A r6 c | j | j dk( ry| j rt | | S | j
|dk s|| j
kD r | j
}| j j | }|s|r| j
|S | j
:| xj
t | z c_ | j
s| j
|S )Nr r6 r )rR rh rn r rO rp r~ r= )rA limitr r s r) rO zHTTPResponse.readline s 77?dllf 4 <<7#E* * ;; " UT[[5HKKE!!%(%
[[
$ KK3v; &K;; "
r6 c | j }||dk( ryd|cxk r|k sn |}| j j | }| xj t | z c_ |s t d |S r )r rR r ro r= r )rA rB ro r s r) r zHTTPResponse._read1_chunked sj ))+
aQ$*$Aww}}Q 3t9$ % % r6 c | j }|y| j j | d | S # t $ r Y yw xY w)Nr6 )r r rR r )rA rB ro s r) r zHTTPResponse._peek_chunked sP --/J ww||J' 44
s 3 ??c 6 | j j S r )rR filenor s r) r zHTTPResponse.fileno s ww~~ r6 c | j
t | j j | xs |}t |t s t |d s|S dj
| S )ax Returns the value of the header matching *name*.
If there are multiple matching headers, the values are
combined into a single string separated by commas and spaces.
If no matching header is found, returns *default* or None if
the *default* is not specified.
If the headers are unknown, raises http.client.ResponseNotReady.
__iter__z, )rS r get_all
isinstancery hasattrrX )rA r' defaultrS s r) getheaderzHTTPResponse.getheader sW << "$ $,,&&t,7 gs #77J+GN99W% %r6 c t | j
t t | j j S )z&Return list of (header, value) tuples.)rS r listr r s r)
getheaderszHTTPResponse.getheaders s- << "$ $DLL&&())r6 c | S r rJ r s r) r zHTTPResponse.__iter__
s r6 c | j S )aj Returns an instance of the class mimetools.Message containing
meta-information associated with the URL.
When the method is HTTP, these headers are those returned by
the server at the head of the retrieved HTML page (including
Content-Length and Content-Type).
When the method is FTP, a Content-Length header will be
present if (as is now usual) the server passed back a file
length in response to the FTP retrieval request. A
Content-Type header will be present if the MIME type can be
guessed.
When the method is local-file, returned headers will include
a Date representing the file's last-modified time, a
Content-Length giving file size, and a Content-Type
containing a guess at the file's type. See also the
description of the mimetools module.
)rS r s r) infozHTTPResponse.info s * ||r6 c | j S )aZ Return the real URL of the page.
In some cases, the HTTP server redirects a client to another
URL. The urlopen() function handles this transparently, but in
some cases the caller needs to know which URL the client was
redirected to. The geturl() method can be used to get at this
redirected URL.
)rt r s r) geturlzHTTPResponse.geturl& s
xxr6 c | j S )zuReturn the HTTP status code that was sent with the response,
or None if the URL is not an HTTP URL.
)rl r s r) getcodezHTTPResponse.getcode2 s
{{r6 )r NNr ))"rG rH rI ru r r r r~ r r r r r r r r r r r r r r r rO r r r r r r r r r
__classcell__r s @r) r r s #< 'BF#P:
#
&P D 0;(:0I6& " 5 &(*
.
r6 r c t j }| dk( r|j dg |j d|_ |S )Nr zhttp/1.1T)ssl _create_default_https_contextset_alpn_protocolspost_handshake_auth) http_versioncontexts r) _create_https_contextr : sC //1Gr""J<0"".&*# Nr6 c
e Zd ZdZdZeZeZdZ dZ
e d Z e d Z
dej ddfd Zd"d
Zd Zd Zd
Zd Zd Zd Zd Zd Zd Zd Zd#dZ d$dZ d Z d Z d Z d Z!d Z"d%dd d Z#di fdd d Z$d Z%d! Z&y)&r r zHTTP/1.1r; r c 6 t | t j S )zFTest whether a file-like object is a text or a binary stream.
)r r
TextIOBase)streams r)
_is_textIOzHTTPConnection._is_textIOQ s &"--00r6 c | |j t v ryyt | d ry t | }|j S # t
$ r Y nw xY wt
| t r t | S y)a Get the content-length based on the body.
If the body is None, we set Content-Length: 0 for methods that expect
a body (RFC 7230, Section 3.3.2). We also set the Content-Length for
any method if the body is a str or bytes-like object and not a file.
Nr r ) upper_METHODS_EXPECTING_BODYr r nbytes TypeErrorr ry r= )bodyrs mvs r) _get_content_lengthz"HTTPConnection._get_content_lengthW sq < ||~!88 4 D!B99 dC t9 s A A A N c R || _ || _ || _ d | _ g | _ d | _ t | _ d | _ d | _ d | _
i | _ d | _ | j || \ | _ | _ | j! | j t" j$ | _ y r )timeoutsource_address blocksizerr _buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterh _tunnel_host _tunnel_port_tunnel_headers_raw_proxy_headers
_get_hostporthostport_validate_hostsocketcreate_connection_create_connection)rA r r r r
r s r) ru zHTTPConnection.__init__w s ," ! "& !%!3!3D$!? DI DII& #)":": r6 c | j r t d | j || \ | _ | _ |r|j | _ n| j j t d | j D sI| j j d j d }d|| j fz | j d< yy)a Set up host and port for HTTP CONNECT tunnelling.
In a connection that uses HTTP CONNECT tunnelling, the host passed to
the constructor is used as a proxy server that relays all communication
to the endpoint passed to `set_tunnel`. This done by sending an HTTP
CONNECT request to the proxy server when the connection is established.
This method must be called before the HTTP connection has been
established.
The headers argument should be a mapping of extra HTTP headers to send
with the CONNECT request.
As HTTP/1.1 is used for HTTP CONNECT tunnelling request, as per the RFC
(https://tools.ietf.org/html/rfc7231#section-4.3.6), a HTTP Host:
header must be provided, matching the authority-form of the request
target provided as the destination for the CONNECT request. If a
HTTP Host: header is not provided via the headers argument, one
is generated and transmitted automatically.
z.Can't set up tunnel for established connectionc 3 B K | ] }|j d k( yw)r Nr< ).0headers r) z,HTTPConnection.set_tunnel.. s O:N6<<>V+:Ns idnaasciiz%s:%dHostN) rr RuntimeErrorr r r copyr clearanyr rY )rA r r rS encoded_hosts r)
set_tunnelzHTTPConnection.set_tunnel s , 99 OP P/3/A/A$/M,4, #*<<>D & & (O$:N:NOO ,,33F;BB7KL+2 d / /61 ,1D ( Pr6 c H |K|j d }|j d }||kD r t ||dz d }|d | }n | j }|r|d dk( r
|d dk( r|dd }||fS # t $ r/ ||dz d dk( r
| j }nt d||dz d z Y dw xY w) Nr: ]r; rx znonnumeric port: '%s'r [r )rfindr r} default_portr )rA r r r js r) r z HTTPConnection._get_hostport s <
3A
3A1uO tAaCDz?D BQx(( DGsNtBx3":Dd| " OAaCDzR' # 0 0 ()@4!:)M NN Os A) )5B! B!c || _ y r )rg )rA levels r) set_debuglevelz HTTPConnection.set_debuglevel s r6 c . d|v r|d dk7 rd|z dz S |S )N :r [ r/ r0 rJ )rA ips r)
_wrap_ipv6zHTTPConnection._wrap_ipv6 s( 2:"Q%7*"9t# # r6 c t j | j rt d| j d| j | j j d | j | j j d fz }|g}| j j D ]o \ }}|j d }|j d }t | st d| t | rt d| |j d||fz q |j d | j d
j | ~| j | j | j" } |j% \ }} }
t' |j( | _ | j, d kD r+| j* D ] }t/ d
|j1 | t2 j4 j6 k7 r/| j9 t; d| d|
j= |j9 y # |j9 w xY w)Nz-Tunnel host can't contain control characters s CONNECT %s:%d %s
r$ r% r Invalid header name Invalid header value s %s: %s
rM r6 rs r r zTunnel connection failed: ) !_contains_disallowed_url_pchar_researchr r} r: r r
_http_vsn_strr r _is_legal_header_name_is_illegal_header_valuer@ sendrX response_classrr rh r rT rR r rg rz rY http
HTTPStatusOKr OSErrorr ) rA connectrS r" r header_bytes value_bytesresponserk r messages r) _tunnelzHTTPConnection._tunnel s , 3 3D4E4E F $ 1 1 4 5
5) OOD - - 4 4V < = % %g .-0 0 ) ! 1 1 7 7 9MFE!==3L ,,y1K( 6 \!KLL' 4 k!KLL NN=L++FF G : w
#((7#$ &&tyy &F '/'<'<'> $WdG&3HKK&@D #" " 5 5F)V]]_5 6 t)))
:4&'--/AR STT *
NN HNN s
4B4H: :I c H | j t | j S dS )z
Returns a dictionary with the headers of the response
received from the proxy server to the CONNECT request
sent to set the tunnel.
If the CONNECT request was not sent, the method returns None.
N)r r` r s r) get_proxy_response_headersz)HTTPConnection.get_proxy_response_headers s/ &&2
7 7 8
r6 c t j d| | j | j | j | j | j f| j
| j | _ | j j t j t j d | j r| j! yy# t $ r(}|j t j k7 r Y d}~Jd}~ww xY w)z3Connect to the host and port specified in __init__.zhttp.client.connectr; N)sysauditr r r r r
rr
setsockoptr IPPROTO_TCP TCP_NODELAYrJ errno ENOPROTOOPTr rP rA es r) rK zHTTPConnection.connect s 'tyy$))D++
YYtyy !4<<1D1DF II !3!3V5G5G K
LLN ww%+++ , s 09C C8 C33C8c t | _ | j }|rd| _ |j | j }|rd| _ |j yy# | j }|rd| _ |j w w xY w)z(Close the connection to the HTTP server.N)r r rr r r )rA rr rN s r) r zHTTPConnection.close sr !99D
H"& H"& s %A (Bc | j '| j r| j n
t | j dkD rt dt
| t |d r| j dkD r t d | j | }|r| j dkD r t d |j | j x}rc|r|j d }t j d| | | j j | |j | j x}rcyt j d| | | j j | y# t $ r` t! |t" j$ j& r$|D ] }| j j | Y yt d t) | z w xY w)
zSend `data' to the server.
``data`` can be a string object, a bytes object, an array object, a
file-like object that supports a .read() method, or an iterable object.
Nr zsend:r zsending a readable encoding file using iso-8859-1rV zhttp.client.sendz9data should be a bytes-like object or an iterable, got %r)rr auto_openrK r rg rz r{ r r r r r rT rU sendallr r collectionsabcIterabletype)rA r& r datablockds r) rE zHTTPConnection.send" s~ 99 ~~ "n$ ??Q '4: & 4 "*+__T*F$//A-67 #yy 88)8 ) 0 0 >I ,dI> !!), $yy 88)8
$dD1 G II d # G$ 8 89 AII%%a( !9;?:!F G G Gs E AG 1G c : | j j | y)zuAdd a line of output to the current request buffer.
Assumes that the line does *not* end with \r\n.
N)r r@ )rA r s r) _outputzHTTPConnection._outputG s
A r6 c # @ K | j dkD r t d | j | }|r| j dkD r t d |j | j x}r6|r|j d }| |j | j x}r5y y w)Nr zreading a readabler_ rV )rg rz r r r r )rA r r rf s r) _read_readablez HTTPConnection._read_readableN s ??Q & ' * doo) 2 3#== 88i8 % , ,\ : O $== 88i8s BB B Fc | j j d dj | j }| j dd= | j | |t |d r| j | }n t
| |f}|D ]e }|s| j dkD r t d |r3| j dk( r$t | d d
j d |z dz }| j | g |r"| j dk( r| j d yyyy# t $ r4 t | }n$# t $ r t dt | z w xY wY w xY w)
zSend the currently buffered request and clear the buffer.
Appends an extra \r\n to the buffer.
A message_body may be specified, to be appended to the request.
)r6 r6 rM Nr zAmessage_body should be a bytes-like object or an iterable, got %rr zZero length chunk ignoredr Xz
r% s 0
)r extendrX rE r rk r r iterre rg rz _http_vsnr= r )rA message_bodyencode_chunkedri chunkschunks r) _send_outputzHTTPConnection._send_outputY s`
J'll4<<( LLO # # |V, ,,\:-
|, +_F * 9: !dnn&: "5z!nD 1 8 8 AE I! "E % $..B"6 ,' #7~O $ ! >>!%l!3$ > ' )H*.|*<)= > >> >s* 8 D E D,+E,!E
EEc z | j r!| j j rd| _ | j t k( r t | _ nt | j | j
| || _ |xs d}| j | |d|d| j }| j | j | | j dk( r&|s
d}|j d rt | \ }}}}}|r. |j d }| j# d t% | n| j& r| j& } | j( }
n| j* } | j, }
| j d } | j/ | } d
| v r t% | } |
| j0 k( r| j# d | n(| j3 d } | j# d | d
|
|s| j# d d yyy# t $ r |j d }Y w xY w# t $ r | j d } Y w xY w)
a` Send a request to the server.
`method' specifies an HTTP request method, e.g. 'GET'.
`url' specifies the object being requested, e.g. '/index.html'.
`skip_host' if True does not add automatically a 'Host:' header
`skip_accept_encoding' if True does not add automatically an
'Accept-Encoding:' header
N/r? r rx rG r% r$ r& r: zAccept-Encodingidentity)r r r r _CS_REQ_STARTEDr _validate_methodrh _validate_pathrB ri _encode_requestrp r2 r r r putheaderr5 r r r r r: r1 rY ) rA rs rt skip_hostskip_accept_encodingrequestnetlocnil
netloc_encr r host_encs r)
putrequestzHTTPConnection.putrequest s ??t 7 7 9 "DO* <<8 #*DL#DLL1 1 f% jS C &T-?-?@ T))'23 >>R >>&)19# .Cc3;%+]]7%;
NN6+t 0 00 vx8#+??7#; v(D/IJ ( 0*= (
c . ;%+]]6%:
; . 7#';;v#67s$ ?G< *H <HH H:9H:c $ |j d S )Nr% )r )rA r s r) r| z HTTPConnection._encode_request s ~~g&&r6 c r t j | }|r t d|d|j d y)z&Validate a method name for putrequest.z)method can't contain control characters. (found at least )N)$_contains_disallowed_method_pchar_rerA r} group)rA rs matchs r) rz z HTTPConnection._validate_method sI 5;;FC ?z J'',{{}&7q:;
; r6 c r t j | }|r t d|d|j d y)z Validate a url for putrequest.&URL can't contain control characters. r r Nr@ rA r r )rA rt r s r) r{ z HTTPConnection._validate_path sL 288= EcW M005
/@ C D
D r6 c r t j | }|r t d|d|j d y)z9Validate a host so it doesn't contain control characters.r r r Nr )rA r r s r) r z HTTPConnection._validate_host# sL 288> EdX N005
/@ C D
D r6 c | j t k7 r
t t |d r|j d }t | st
d| t | }t | D ]r \ }}t |d r|j d ||< n-t |t r t | j d ||< t || sct
d|| dj | }|dz |z }| j | y) zkSend a request header line to the server.
For example: h.putheader('Accept', 'text/html')
r r% r< r r= s
s : N)r ry r r r rC r} r enumerater r ry rD rX ri )rA r" valuesr one_valuer s r) r} zHTTPConnection.putheader+ s
<.[ s )r r r r r rg rF rr rh r ConnectionErrorr rq rj r )rA rN s r) getresponsezHTTPConnection.getresponse s ??t 7 7 9 "DO" <<< '4??"4<<0 0 ??Q **499doo26,, + @H **499T\\*JH
&&(2 22#DL""
O #+O #
NN s$
D$ =E E $D??E E)NN)NF)FFr )'rG rH rI rp rB r rF HTTP_PORTr1 r` rg staticmethodr r r _GLOBAL_DEFAULT_TIMEOUTru r, r r5 r: rP rR rK r rE ri rk ru r r| rz r{ r r} r r r r rJ r6 r) r r G s I M!N LIJ1 1
> #'0N0N $;."1H (
' R
! #GJ 4(l 16(-}~';DD 8 Ge G )-b G $G
,=\=r6 r c R e Zd ZdZeZdej dddd fd Z fdZ xZ
S )HTTPSConnectionz(This class allows communication via SSL.Nr
)r r
r r c r t t | ||||| |t | j }|| _ y )N)r )r r ru r rp _context)rA r r r r
r r r s r) ru zHTTPSConnection.__init__ sB
/4 1$g2@ >$($$ $ S Sr6 r c e Zd Zy)r NrG rH rI rJ r6 r) r r s r6 r c e Zd Zy)r Nr rJ r6 r) r r r6 r c e Zd Zy)r Nr rJ r6 r) r r r r6 r c e Zd Zd Zy)r c " |f| _ || _ y r )argsrk )rA rk s r) ru zUnknownProtocol.__init__ s H r6 NrG rH rI ru rJ r6 r) r r s r6 r c e Zd Zy)r Nr rJ r6 r) r r r r6 r c e Zd Zy)r
Nr rJ r6 r) r
r
r r6 r
c 2 e Zd ZddZd Zej Zy)r Nc 0 |f| _ || _ || _ y r )r partialexpected)rA r r s r) ru zIncompleteRead.__init__ s H
r6 c | j d| j z }nd}d| j j t | j |fz S )Nz, %i more expectedrx z%s(%i bytes read%s))r r rG r= r r[ s r) __repr__zIncompleteRead.__repr__ sK == $$t}}4AA$(?(?(+DLL(91(> > >r6 r )rG rH rI ru r r" __str__rJ r6 r) r r s !> nnGr6 r c e Zd Zy)r
Nr rJ r6 r) r
r
r r6 r
c e Zd Zy)r Nr rJ r6 r) r r r r6 r c e Zd Zy)r Nr rJ r6 r) r r r r6 r c e Zd Zy)r Nr rJ r6 r) r r r r6 r c e Zd Zd Zy)r c < |s t | }|f| _ || _ y r )r{ r rE r s r) ru zBadStatusLine.__init__ s :DE r6 Nr rJ r6 r) r r s r6 r c e Zd Zd Zy)r c B t j | dt |fz y )Nz&got more than %d bytes when reading %s)r ru rP )rA line_types r) ru zLineTooLong.__init__! s" t%M(0)'<&= >r6 Nr rJ r6 r) r r s >r6 r c e Zd Zd Zy)r c b t j | d t j | g|i | y )Nrx )r ru ConnectionResetError)rA poskws r) ru zRemoteDisconnected.__init__& s* tR( %%d7S7B7r6 Nr rJ r6 r) r r % s 8r6 r )r& )Ir email.parserrZ
email.messagerY rG r rer rT collections.abcrb urllib.parser __all__r r rj r ry r globalsupdaterH __members__r phraser rP rQ r compile fullmatchrC rA rD r@ r r r* r r5 rO Messager8 rT r` rb BufferedIOBaser r r r r r@ ImportError Exceptionr r r r r r
r r
r r r r r r r r )vs 0r) r s DL
!
# ,, - #'//"="="D"D"F G"FQQ["F G 8 #
#67AA %2::&CDKK %/BJJ/@$A ! (2rzz-'@ $ 3 E % %--'' <$ .9 @ ) 0v2$$ vr
} } ~!&S. S8 NN$% I
= m
m M ] m / . . M >- >
8-} 8 -
Hx* s
H#%H( (H0/H0