U ÀÒ.eÅ0 ã @ s€ d Z ddlZddlZddlZddlmZ ddlmZmZm Z ddl mZ ddlm Z ddlmZ dd lmZ G d d„ deƒZdS )a Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco é Né )ÚCharSetGroupProber)Ú InputStateÚLanguageFilterÚProbingState)ÚEscCharSetProber)ÚLatin1Prober)ÚMBCSGroupProber)ÚSBCSGroupProberc @ sn e Zd ZdZdZe d¡Ze d¡Ze d¡Z dddd d ddd dœZ ejfdd„Z dd„ Zdd„ Zdd„ ZdS )ÚUniversalDetectoraq The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result gš™™™™™É?s [€-ÿ]s (|~{)s [€-Ÿ]zWindows-1252zWindows-1250zWindows-1251zWindows-1256zWindows-1253zWindows-1255zWindows-1254zWindows-1257)z iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8z iso-8859-9ziso-8859-13c C sN d | _ g | _d | _d | _d | _d | _d | _|| _t t ¡| _d | _| ¡ d S )N)Ú_esc_charset_proberÚ_charset_probersÚresultÚdoneÚ _got_dataÚ_input_stateÚ _last_charÚlang_filterÚloggingZ getLoggerÚ__name__ÚloggerÚ_has_win_bytesÚreset)Úselfr © r úI/usr/lib/python3.8/site-packages/pip/_vendor/chardet/universaldetector.pyÚ__init__Q s zUniversalDetector.__init__c C sV ddddœ| _ d| _d| _d| _tj| _d| _| jr>| j ¡ | j D ]}| ¡ qDdS )zæ Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. Nç ©ÚencodingÚ confidenceÚlanguageFó )r r r r r Ú PURE_ASCIIr r r r r )r Úproberr r r r ^ s zUniversalDetector.resetc C s> | j r dS t|ƒsdS t|tƒs(t|ƒ}| jsÞ| tj¡rJddddœ| _nv| tj tj f¡rlddddœ| _nT| d¡r†ddddœ| _n:| d ¡r d dddœ| _n | tjtjf¡rÀddddœ| _d| _| jd dk rÞd| _ dS | j tjk�r.| j |¡�rtj| _ n*| j tjk�r.| j | j| ¡�r.tj| _ |dd… | _| j tjk�r–| j�s^t| jƒ| _| j |¡tjk�r:| jj| j ¡ | jjdœ| _d| _ n¤| j tjk�r:| j�sät | jƒg| _| jt!j"@ �rÖ| j #t$ƒ ¡ | j #t%ƒ ¡ | jD ]:}| |¡tjk�rê|j| ¡ |jdœ| _d| _ �q&�qê| j& |¡�r:d| _'dS )aý Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIGç ð?Ú r zUTF-32s þÿ zX-ISO-10646-UCS-4-3412s ÿþzX-ISO-10646-UCS-4-2143zUTF-16Tr éÿÿÿÿ)(r ÚlenÚ isinstanceÚ bytearrayr Ú startswithÚcodecsÚBOM_UTF8r ÚBOM_UTF32_LEÚBOM_UTF32_BEÚBOM_LEÚBOM_BEr r r# ÚHIGH_BYTE_DETECTORÚsearchÚ HIGH_BYTEÚESC_DETECTORr Z ESC_ASCIIr r r Úfeedr ZFOUND_ITÚcharset_nameÚget_confidencer! r r r ZNON_CJKÚappendr r ÚWIN_BYTE_DETECTORr )r Zbyte_strr$ r r r r6 o sŽ þ ÿþ þ þ þ ÿû þ zUniversalDetector.feedc C st | j r| jS d| _ | js&| j d¡ n²| jtjkrBddddœ| _n–| jtjkrØd}d}d}| j D ]"}|sjq`| ¡ }||kr`|}|}q`|rØ|| jkrØ|j}|j ¡ }| ¡ }| d ¡rÈ| jrÈ| j ||¡}|||jdœ| _| j ¡ tjk�rn| jd dk�rn| j d¡ | j D ]`}|�s�qt|tƒ�rP|jD ] }| j d|j|j| ¡ ¡ �q,n| j d|j|j| ¡ ¡ �q| jS ) zæ Stop analyzing the current document and come up with a final prediction. :returns: The ``result`` attribute, a ``dict`` with the keys `encoding`, `confidence`, and `language`. Tzno data received!Úasciir% r&