U
e5d @ sf d Z ddlZddlZddlZddlZddlZddlZddlZ ddl
m Z dddddd d
d d d
ddddddddgZ dZ
dZdZdZdZdZe ejj d d ejj D Zd Zd Zed j Z ed!j Z ed"Z ed#Z!d$d%d&hZ"dBd(d)Z#G d*d+ d+ej$j%Z&d,d- Z'e&fd.d/Z(G d0d dej)Z*G d1d dZ+z ddl,Z,W n e-k
r` Y n X G d2d3 d3e+Z.e /d3 G d4d de0Z1G d5d de1Z2G d6d d e1Z3G d7d de1Z4G d8d d e1Z5G d9d
d
e1Z6G d:d d e1Z7G d;d
d
e1Z8G dd de8Z;G d?d de1Ze<Z?e1Z@dS )Ca
HTTP/1.1 client library
HTTPConnection goes through a number of "states", which define when a client
may legally make another request or fetch the response for a particular
request. This diagram details these state transitions:
(null)
|
| HTTPConnection()
v
Idle
|
| putrequest()
v
Request-started
|
| ( putheader() )* endheaders()
v
Request-sent
|\_____________________________
| | getresponse() raises
| response = getresponse() | ConnectionError
v v
Unread-response Idle
[Response-headers-read]
|\____________________
| |
| response.read() | putrequest()
v v
Idle Req-started-unread-response
______/|
/ |
response.read() | | ( putheader() )* endheaders()
v v
Request-started Req-sent-unread-response
|
| response.read()
v
Request-sent
This diagram presents the following rules:
-- a second request may not be started until {response-headers-read}
-- a response [object] cannot be retrieved until {request-sent}
-- there is no differentiation between an unread response body and a
partially read response body
Note: this enforcement is applied by the HTTPConnection class. The
HTTPResponse class does not enforce this state machine, which
implies sophisticated clients may accelerate the request/response
pipeline. Caution should be taken, though: accelerating the states
beyond the above pattern may imply knowledge of the server's
connection-close behavior for certain requests. For example, it
is impossible to tell whether the server will close the connection
UNTIL the response headers have been read; this means that further
requests cannot be placed into the pipeline until it is known that
the server will NOT be closing the connection.
Logical State __state __response
------------- ------- ----------
Idle _CS_IDLE None
Request-started _CS_REQ_STARTED None
Request-sent _CS_REQ_SENT None
Unread-response _CS_IDLE
Req-started-unread-response _CS_REQ_STARTED
Req-sent-unread-response _CS_REQ_SENT
N)urlsplit HTTPResponseHTTPConnection
HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead
InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady
BadStatusLine LineTooLongRemoteDisconnectederror responsesP i ZUNKNOWNZIdlezRequest-startedz Request-sentc C s i | ] }||j qS )phrase).0vr r #/usr/lib64/python3.8/http/client.py
j s r i d s [^:\s][^:\r\n]*s \n(?![ \t])|\r(?![ \t\n])z[ - ]z[ - ]ZPATCHZPOSTZPUTdatac
C sh z | dW S tk
rb } z8t|j|j|j|jd| | |j|j |f dW 5 d}~X Y nX dS )zd? Z Z!S )Fr r Nc C sR | d| _|| _|| _d | _| _t| _t| _t| _ t| _
t| _ t| _ t| _
d S )Nrb)Zmakefiler@
debuglevel_methodrA msg_UNKNOWNversionstatusreasonchunked
chunk_leftlength
will_close)r1 sockrK methodurlr r r __init__ s zHTTPResponse.__init__c C s t | jtd d}t|tkr*td| jdkrBtdt| |sNt dz|
d d\}}}W nF t k
r z|
d d\}}d}W n t k
r d}Y nX Y nX | d s|
t|z$t|}|d
k s|d krt|W n t k
r t|Y nX |||fS ) Nr+ rC z status liner zreply:z-Remote end closed connection without response zHTTP/r i )strr@ r= r> r- r rK printreprr split
ValueError
startswith _close_connr int)r1 r3 rO rP rQ r r r _read_status s2
zHTTPResponse._read_statusc C s | j d k rd S | \}}}|tkr&qHt| j}| jdkrDtd| ~q| | _| _| | _
|dkrnd| _ n| drd| _ nt
|t| j | _ | _| jdkr| j D ]\}}td|d | q| j d }|r| d
krd | _d | _nd | _| | _d | _| j d
}|rb| jsbzt|| _W n tk
rL d | _Y nX | jdk rhd | _nd | _|tks|tksd| krdk sn | j dkrd| _| js| js| jd krd | _d S )Nr zheaders:)zHTTP/1.0zHTTP/0.9
zHTTP/1. header:r* transfer-encodingrR TFcontent-lengthr HEAD) rA rd ZCONTINUErB r@ rK r] coderP striprQ rO ra r rI rM itemsgetr, rR rS _check_closerU rT rc r` Z
NO_CONTENTZ NOT_MODIFIEDrL ) r1 rO rP rQ Zskipped_headershdrvalZtr_encrT r r r begin5 sf
zHTTPResponse.beginc C sv | j d}| jdkr.|r*d| kr*dS dS | j dr>dS |rRd| krRdS | j d}|rrd| krrdS dS )NZ
connectionrf closeTFz
keep-alivezproxy-connection)rA ro rO r, )r1 ZconnZpconnr r r rp } s
zHTTPResponse._check_closec C s | j }d | _ | d S N)r@ rt )r1 r@ r r r rb s zHTTPResponse._close_connc s$ zt W 5 | j r | X d S ru )r@ rb superrt r1 __class__r r rt s zHTTPResponse.closec s t | jr| j d S ru )rv flushr@ rw rx r r rz s
zHTTPResponse.flushc C s dS )zAlways returns TrueTr rw r r r readable s zHTTPResponse.readablec C s
| j dkS )z!True if the connection is closed.N)r@ rw r r r isclosed s zHTTPResponse.isclosedc C s | j d krdS | jdkr$| dS |d k rRt|}| |}t|d | S | jr`| S | j d krv| j
}n6z| | j }W n t k
r | Y nX d| _ | |S d S )Nr< rk r )
r@ rL rb bytearrayreadinto
memoryviewtobytesrR _readall_chunkedrT read
_safe_readr
)r1 amtbr2 sr r r r s*
zHTTPResponse.readc C s | j dkrdS | jdkr$| dS | jr4| |S | jdk r^t|| jkr^t|d| j }| j |}|s||r|| n&| jdk r| j|8 _| js| |S )z^Read up to len(b) bytes into bytearray b and return the number
of bytes read.
Nr rk ) r@ rL rb rR _readinto_chunkedrT r- r r~ )r1 r r2 r r r r~ s$
zHTTPResponse.readintoc C sr | j td }t|tkr$td|d}|dkrB|d | }z t|dW S tk
rl | Y nX d S )Nr+ z
chunk size ;r ) r@ r= r> r- r findrc r` rb )r1 r3 ir r r _read_next_chunk_size s
z"HTTPResponse._read_next_chunk_sizec C s: | j td }t|tkr$td|s*q6|dkr q6q d S )Nr+ z trailer liner9 )r@ r= r> r- r r1 r3 r r r _read_and_discard_trailer s z&HTTPResponse._read_and_discard_trailerc C sl | j }|sh|d k r | d z | }W n tk
rD tdY nX |dkrb| | d }|| _ |S )NrZ r< r )rS r r r` r
r rb )r1 rS r r r _get_chunk_left s
z HTTPResponse._get_chunk_leftc C sp | j tkstg }z6| }|d kr&q>|| | d| _qd|W S tk
rj td|Y nX d S Nr r< ) rR rN AssertionErrorr r0 r rS rE r
)r1 valuerS r r r r 8 s z HTTPResponse._readall_chunkedc C s | j tkstd}t|}zv| }|d kr2|W S t||kr\| |}|| | _|| W S |d | }| |}||d }||7 }d| _q W n( tk
r tt |d| Y nX d S )Nr )
rR rN r r r r- _safe_readintorS r
bytes)r1 r Z total_bytesZmvbrS r2 Ztemp_mvbr r r r F s$
z HTTPResponse._readinto_chunkedc C s. | j |}t||k r*t||t| |S )a Read the number of bytes requested.
This function should be used when bytes "should" be present for
reading. If the bytes are truly not available (due to EOF), then the
IncompleteRead exception can be used to detect the problem.
)r@ r r- r
)r1 r r r r r r ^ s zHTTPResponse._safe_readc C s: t |}| j|}||k r6tt|d| || |S )z2Same as _safe_read, but for reading into a buffer.N)r- r@ r~ r
r )r1 r r r2 r r r r j s
zHTTPResponse._safe_readintoc C s | j dks| jdkrdS | jr(| |S | jdk rJ|dk sD|| jkrJ| j}| j |}|sh|rh| n | jdk r| jt|8 _|S )zvRead with at most one underlying system call. If at least one
byte is buffered, return that instead.
Nrk r< r )r@ rL rR _read1_chunkedrT read1rb r- )r1 r2 resultr r r r r s
zHTTPResponse.read1c C s4 | j d ks| jdkrdS | jr(| |S | j |S )Nrk r< )r@ rL rR
_peek_chunkedpeek)r1 r2 r r r r s
zHTTPResponse.peekc s | j d ks| jdkrdS | jr*t |S | jd k rL|dk sF|| jkrL| j}| j |}|sj|rj| n | jd k r| jt|8 _|S )Nrk r< r )r@ rL rR rv r= rT rb r- )r1 limitr rx r r r= s
zHTTPResponse.readlinec C sd | }|d ks|dkr dS d| kr0|ks6n |}| j|}| jt|8 _|s`td|S r )r r@ r rS r- r
)r1 r2 rS r r r r r s zHTTPResponse._read1_chunkedc C sD z | }W n tk
r" Y dS X |d kr0dS | j|d | S )Nr< )r r
r@ r )r1 r2 rS r r r r s zHTTPResponse._peek_chunkedc C s
| j S ru )r@ filenorw r r r r s zHTTPResponse.filenoc C sF | j dkrt | j |p |}t|ts4t|ds8|S d|S dS )ax Returns the value of the header matching *name*.
If there are multiple matching headers, the values are
combined into a single string separated by commas and spaces.
If no matching header is found, returns *default* or None if
the *default* is not specified.
If the headers are unknown, raises http.client.ResponseNotReady.
N__iter__z, )rA r Zget_all
isinstancer\ hasattrrE )r1 r& defaultrA r r r getheader s
zHTTPResponse.getheaderc C s | j dkrt t| j S )z&Return list of (header, value) tuples.N)rA r listrn rw r r r
getheaders s
zHTTPResponse.getheadersc C s | S ru r rw r r r r s zHTTPResponse.__iter__c C s | j S )aj Returns an instance of the class mimetools.Message containing
meta-information associated with the URL.
When the method is HTTP, these headers are those returned by
the server at the head of the retrieved HTML page (including
Content-Length and Content-Type).
When the method is FTP, a Content-Length header will be
present if (as is now usual) the server passed back a file
length in response to the FTP retrieval request. A
Content-Type header will be present if the MIME type can be
guessed.
When the method is local-file, returned headers will include
a Date representing the file's last-modified time, a
Content-Length giving file size, and a Content-Type
containing a guess at the file's type. See also the
description of the mimetools module.
)rA rw r r r info s zHTTPResponse.infoc C s | j S )aZ Return the real URL of the page.
In some cases, the HTTP server redirects a client to another
URL. The urlopen() function handles this transparently, but in
some cases the caller needs to know which URL the client was
redirected to. The geturl() method can be used to get at this
redirected URL.
)rX rw r r r geturl s
zHTTPResponse.geturlc C s | j S )zuReturn the HTTP status code that was sent with the response,
or None if the URL is not an HTTP URL.
)rP rw r r r getcode s zHTTPResponse.getcode)r NN)N)r )r )r )N)"r5 r6 r7 rY rd rs rp rb rt rz r{ r| r r~ r r r r r r r r r r= r r r r r r r r r
__classcell__r r rx r r s<
!H
"
c @ s
e Zd ZdZdZeZeZdZ dZ
e dd Z e dd Z
d ejd d
fd d Zd7d
dZdd Zdd Zdd Zdd Zdd Zdd Zdd Zd d Zd8d d!Zd9d"d#Zd$d% Z d&d' Z d(d) Z d*d+ Z d,d- Z d:d d.d/d0Z!d i fd d.d1d2Z"d3d4 Z#d5d6 Z$d S );r rf zHTTP/1.1r+ r c C s t | tjS )zFTest whether a file-like object is a text or a binary stream.
)r io
TextIOBase)streamr r r
_is_textIO s zHTTPConnection._is_textIOc C sf | dkr | tkrdS dS t| dr*dS zt| }|jW S tk
rN Y nX t| trbt| S dS )a Get the content-length based on the body.
If the body is None, we set Content-Length: 0 for methods that expect
a body (RFC 7230, Section 3.3.2). We also set the Content-Length for
any method if the body is a str or bytes-like object and not a file.
Nr r ) upper_METHODS_EXPECTING_BODYr r nbytes TypeErrorr r\ r- )bodyrW Zmvr r r _get_content_length s
z"HTTPConnection._get_content_lengthN c C sn || _ || _|| _d | _g | _d | _t| _d | _d | _ d | _
i | _ | ||\| _
| _| | j
tj| _d S ru )timeoutsource_address blocksizerV _buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterL _tunnel_host _tunnel_port_tunnel_headers
_get_hostporthostport_validate_hostsocketZcreate_connection_create_connection)r1 r r r r r r r r rY 4 s zHTTPConnection.__init__c C s< | j rtd| ||\| _| _|r.|| _n
| j dS )aD Set up host and port for HTTP CONNECT tunnelling.
In a connection that uses HTTP CONNECT tunneling, the host passed to the
constructor is used as a proxy server that relays all communication to
the endpoint passed to `set_tunnel`. This done by sending an HTTP
CONNECT request to the proxy server when the connection is established.
This method must be called before the HTTP connection has been
established.
The headers argument should be a mapping of extra HTTP headers to send
with the CONNECT request.
z.Can't set up tunnel for established connectionN)rV RuntimeErrorr r r r clear)r1 r r rA r r r
set_tunnelJ s zHTTPConnection.set_tunnelc C s |d kr| d}| d}||krzt||d d }W nH tk
r ||d d dkrh| j}ntd||d d Y nX |d | }n| j}|r|d dkr|d dkr|dd }||fS ) Nr* ]r+ r[ znonnumeric port: '%s'r [r )rfindrc r` default_portr )r1 r r r jr r r r b s
z HTTPConnection._get_hostportc C s
|| _ d S ru )rK )r1 levelr r r set_debuglevelv s z HTTPConnection.set_debuglevelc C s d| j | jf }|d}| | | j D ](\}}d||f }|d}| | q.| d | j| j| jd}| \}} }
| t
j j kr|
td| |
f |jtd } t| tkrtd | sq| d
krq| jd krtd | qd S )
NzCONNECT %s:%d HTTP/1.0
asciiz%s: %s
r r: rW z Tunnel connection failed: %d %sr+ r8 r9 r rg )r r r sendr rn response_classrV rL rd http
HTTPStatusZOKrt OSErrorrm r@ r= r> r- r rK r] rF ) r1 Z connect_strZ
connect_bytesheaderr Z
header_strZ header_bytesresponserO rl messager3 r r r _tunnely s4
zHTTPConnection._tunnelc C sB | | j| jf| j| j| _| jtjtj d | j
r>| dS )z3Connect to the host and port specified in __init__.r+ N) r r r r r rV Z
setsockoptr Z IPPROTO_TCPZ TCP_NODELAYr r rw r r r connect s
zHTTPConnection.connectc C sB t | _z | j}|r d| _| W 5 | j}|r s z/HTTPConnection._send_request..r r+ r zaccept-encodingr ri rh Fr z Unable to determine size of %rTzTransfer-EncodingrR zContent-Lengthr r ) frozensetr r rK r] r r\ rn r r( r ) r1 rW rX r rA r Z header_namesZskipsZcontent_lengthrq r r r r r s0
z HTTPConnection._send_requestc C s | j r| j rd| _ | jtks&| j r0t| j| jdkrR| j| j| j| jd}n| j| j| jd}z\z | W n t
k
r | Y nX |j t
kstt| _|j r| n|| _ |W S | Y nX dS )a) Get the response from the server.
If the HTTPConnection is in the correct state, returns an
instance of HTTPResponse or of whatever object is returned by
the response_class variable.
If a request has not been sent or if a previous response has
not be handled, ResponseNotReady is raised. If the HTTP
response indicates that the connection should be closed, then
it will be closed before the response is returned. When the
connection is closed, the underlying socket is closed.
Nr r )r r| r r r rK r rV rL rs ConnectionErrorrt rU rN r r )r1 r r r r getresponse s0
zHTTPConnection.getresponse)NN)NF)FF)N)%r5 r6 r7 r r r r HTTP_PORTr r rK staticmethodr r r _GLOBAL_DEFAULT_TIMEOUTrY r r r r r rt r r r r r r r r r r r r r r r r r r r sL
&
6
.c sH e Zd ZdZeZdddejdfdddd fdd Z fddZ Z
S ) HTTPSConnectionz(This class allows communication via SSL.Nr )contextcheck_hostnamer c s t t| j||||| d |d k s2|d k s2|d k rHdd l}
|
dtd || _|| _|d krtt }|j
d k rtd|_
|j tj k} |d kr|j
}|r| std|s|r||| |j
d k rd|_
|| _|d k r|| j_
d S )N)r r zTkey_file, cert_file and check_hostname are deprecated, use a custom context instead.rZ TzMcheck_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED)rv r rY warningswarnDeprecationWarningkey_file cert_filesslZ _create_default_https_contextZpost_handshake_authZ verify_modeZ CERT_NONEr r` Zload_cert_chain_context) r1 r r r r r r r r r r Z will_verifyrx r r rY c s<
zHTTPSConnection.__init__c s6 t | jr| j}n| j}| jj| j|d| _dS )z(Connect to a host on a given (SSL) port.)server_hostnameN)rv r r r r Z wrap_socketrV )r1 r rx r r r s
zHTTPSConnection.connect) r5 r6 r7 __doc__
HTTPS_PORTr r r rY r r r r rx r r \ s $r c @ s e Zd ZdS )r Nr5 r6 r7 r r r r r s c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd Zdd ZdS )r c C s |f| _ || _d S ru )argsrO )r1 rO r r r rY s zUnknownProtocol.__init__Nr5 r6 r7 rY r r r r r s c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd ZdS )r Nr r r r r r s c @ s$ e Zd ZdddZdd ZejZdS )r
Nc C s |f| _ || _|| _d S ru )r partialexpected)r1 r" r# r r r rY s zIncompleteRead.__init__c C s2 | j d k rd| j }nd}d| jjt| j|f S )Nz, %i more expectedr[ z%s(%i bytes read%s))r# ry r5 r- r" )r1 er r r __repr__ s
zIncompleteRead.__repr__)N)r5 r6 r7 rY r% r" __str__r r r r r
s
c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd ZdS )r
Nr r r r r r
s c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd ZdS )r Nr r r r r r s c @ s e Zd Zdd ZdS )r c C s |s t |}|f| _|| _d S ru )r^ r r3 r r r r rY s zBadStatusLine.__init__Nr! r r r r r s c @ s e Zd Zdd ZdS )r c C s t | dt|f d S )Nz&got more than %d bytes when reading %s)r rY r> )r1 Z line_typer r r rY s zLineTooLong.__init__Nr! r r r r r s c @ s e Zd Zdd ZdS )r c O s" t | d tj| f|| d S )Nr[ )r rY ConnectionResetError)r1 poskwr r r rY s zRemoteDisconnected.__init__Nr! r r r r r s )r )Ar Z email.parserrG Z
email.messager r rer Zcollections.abcr Z urllib.parser __all__r r rN r r r globalsupdater __members__r r r> r? compile fullmatchr r r r r r r( r ZMessager) rB rI BufferedIOBaser r r ImportErrorr r0 Exceptionr r r r r r r
r r
r r r r r' r r r r r r s F
W 8