3
fN @ s d Z ddlZed\ZZZZZejdej ej
B j Z ejdej j
Zejdej ejB j
Zejdej j
Zejdej j
Zejd ej j
ZG d
d d eZejed d
Zejdd dD ejdd dD ejdd dD G dd dZedkrddlm Z e ddd dS ) a Define partial Python code Parser used by editor and hyperparser.
Instances of ParseMap are used with str.translate.
The following bound search and match functions are defined:
_synchre - start of popular statement;
_junkre - whitespace or comment line;
_match_stringre: string, possibly without closer;
_itemre - line that may have bracket structure start;
_closere - line that must be followed by dedent.
_chew_ordinaryre - non-special characters.
N z
^
[ \t]*
(?: while
| else
| def
| return
| assert
| break
| class
| continue
| elif
| try
| except
| raise
| import
| yield
)
\b
z'
[ \t]*
(?: \# \S .* )?
\n
aK
\""" [^"\\]* (?:
(?: \\. | "(?!"") )
[^"\\]*
)*
(?: \""" )?
| " [^"\\\n]* (?: \\. [^"\\\n]* )* "?
| ''' [^'\\]* (?:
(?: \\. | '(?!'') )
[^'\\]*
)*
(?: ''' )?
| ' [^'\\\n]* (?: \\. [^'\\\n]* )* '?
zM
[ \t]*
[^\s#\\] # if we match, m.end()-1 is the interesting char
z_
\s*
(?: return
| break
| continue
| raise
| pass
)
\b
z
[^[\](){}#'"\\]+
c @ s e Zd ZdZdd ZdS )ParseMapap Dict subclass that maps anything not in dict to 'x'.
This is designed to be used with str.translate in study1.
Anything not specifically mapped otherwise becomes 'x'.
Example: replace everything except whitespace with 'x'.
>>> keepwhite = ParseMap((ord(c), ord(c)) for c in ' \t\n\r')
>>> "a + b\tc\nd".translate(keepwhite)
'x x x\tx\nx'
c C s dS )Nx )selfkeyr r 4/opt/alt/python36/lib64/python3.6/idlelib/pyparse.py __missing__r s zParseMap.__missing__N)__name__
__module__ __qualname____doc__r r r r r r f s
r r c c s | ]}t |t d fV qdS )(N)ord).0cr r r x s r z({[c c s | ]}t |t d fV qdS ))N)r )r r r r r r y s z)}]c c s | ]}t |t |fV qd S )N)r )r r r r r r z s z"'\
#c @ s e Zd Zdd Zdd ZdefddZdd Zd
d Zd d
Z dd Z
dd Z dd Z dd Z
dd Zdd Zdd Zd d ZdS ) Parserc C s || _ || _d S )N) indentwidthtabwidth)r r r r r r __init__ s zParser.__init__c C s || _ d| _d S )Nr )code study_level)r sr r r set_code s zParser.set_codeNc
C s | j d }}|sdS t|}xftdD ]Z}|jdd|}|dk rBP |jdd|d }||||}|r|||j r||j }P |}q&W |dkr||}|r||j r|j }|S |d }x.|||}|r|j \} }|| s| }qP qW |S )a^
Return index of a good place to begin parsing, as close to the
end of the string as possible. This will be the start of some
popular stmt like "if" or "def". Return None if none found:
the caller should pass more prior context then, if possible, or
if not (the entire program text up until the point of interest
has already been tried) pass 0 to set_lo().
This will be reliable iff given a reliable is_char_in_string()
function, meaning that when it says "no", it's absolutely
guaranteed that the char is not in a string.
Nr z:
r
)r lenrangerfindstartspan)
r Zis_char_in_string_synchrer poslimitZtriesimr r r r find_good_parse_start s8
z Parser.find_good_parse_startc C s |dkr| j |d | _ dS )zx Throw away the start of the string.
Intended to be called with the result of find_good_parse_start().
r N)r )r lor r r set_lo s z
Parser.set_loc
C s | j dkrdS d| _ | j}|jt}|jdd}|jdd}|jdd}|jdd}|jdd}t}d }}d g | _}|j}d t| }}x||k rn|| } |d }| dkrq| dkr|d }|d kr|| q| d
kr|d }q| d kr |r|d }q| d ks| d
kr$| }
||d |d |
d kr>|
d }
|} t|
d } || }x||k r|| } |d }| dkrqX||d || |
kr|| }P | dkr|d }| d krX|d kr|| P qX| dkrX|| dkr|d }|d }qXqXW |d | kr t }qt
}q| dkr<|j d|}q|| dkrd|d }|d |krdt }|d }qW |t kr|t
kr|d krt
}|| _|d |kr|| dS )zFind the line numbers of non-continuation lines.
As quickly as humanly possible , find the line numbers (0-
based) of the non-continuation lines.
Creates self.{goodlines, continuation}.
r NZxxxxxxxxxZxxxxZxxz
xr r r r "' \#)r r translatetransreplaceC_NONE goodlinesappendr C_STRING_FIRST_LINEC_STRING_NEXT_LINESfind C_BACKSLASH C_BRACKET continuation)
r r r? levelZlnor8 Z push_goodr' nchZquoteZfirstlnowr r r _study1 s
zParser._study1c C s | j | jS )N)rD r? )r r r r get_continuation_typeT s z Parser.get_continuation_typec C s | j dkrdS | j d| _ | j| j }}t|d }t|}xX|r|}x4t||d || D ]}|jdd|d d }q`W t||r|d }q@P q@W |dkr|}|| | _| _ d}g }|j
} |dfg}
x||k rxt |||} | rD| j } | d }x&||kr || dkr |d } qW ||kr4|| }| }||krDP || }
|
dkr~| | |
j
|t|f |
}|d }q|
d kr|r|d
= |
}|d }|
j
|t|f q|
d
ks|
d kr|
j
|t|d f |
}t
|||j }|
j
|t|f q|
d krL|
j
|t|d f |jd||d }|
j
|t|f q|d }|| dkrn|
|| }|d }qW || _|r|d nd| _t|
| _dS )am
study1 was sufficient to determine the continuation status,
but doing more requires looking at every character. study2
does this for the last interesting statement in the block.
Creates:
self.stmt_start, stmt_end
slice indices of last interesting stmt
self.stmt_bracketing
the bracketing structure of the last interesting stmt; for
example, for the statement "say(boo) or die",
stmt_bracketing will be ((0, 0), (0, 1), (2, 0), (2, 1),
(4, 0)). Strings and comments are treated as brackets, for
the matter.
self.lastch
last interesting character before optional trailing comment
self.lastopenbracketpos
if continuation is C_BRACKET, index of last open bracket
r/ Nr r r z
z([{z)]}r- r. r2 r3 r3 )r rD r r8 r r r! _junkre
stmt_startstmt_endr9 _chew_ordinaryreend_match_stringrer< lastchlastopenbracketpostuplestmt_bracketing)r r r8 r' pqZnothingrM stackZ
push_stackZ
bracketingr( ZnewprB r r r _study2X s
zParser._study2c C s | j | j}| j}t|}|jdd|d }}|d }xj||k r|t||}|rf|j d }d}P q<|jd|d }}q!z\s*\\Nz
) rT r rH r< rL rK rematchr rV r )r r r' startposendposfoundr@ rB r r r compute_backslash_indent sH
z Parser.compute_backslash_indentc C sN | j | j| j }}|}| j}x ||k r@|| dkr@|d }q"W ||| S )z`Return the leading whitespace on the initial line of the last
interesting stmt.
z r )rT rH rI r )r r' rA rW r r r r get_base_indent_string5 s z Parser.get_base_indent_stringc C s | j | jdkS )z=Return True if the last interesting statemtent opens a block.:)rT rM )r r r r is_block_openerA s zParser.is_block_openerc C s | j t| j| jdk S )z=Return True if the last interesting statement closes a block.N)rT _closerer rH )r r r r is_block_closerF s zParser.is_block_closerc C s | j | jS )zReturn bracketing structure of the last interesting statement.
The returned tuple is in the format defined in _study2().
)rT rP )r r r r get_last_stmt_bracketingK s z Parser.get_last_stmt_bracketing)r
r r r r r$ r) r+ rD rE rT rX rZ ra rb rd rf rg r r r r r } s < {
9 r __main__)mainz idlelib.idle_test.test_pyparser/ ) verbosity) r
r\ r r7 r= r: r; r> compileVERBOSE MULTILINEsearchr$ r] rG DOTALLrL rU re rJ dictr fromkeysr5 updater r
Zunittestri r r r r s4 Z