3
\m @ s d Z dZddlZddlZddlmZ dZi add ZG dd d e Z
G d
d d e
Z G d d
d
e
Z dd Z
G dd de ZG dd deZG dd deZG dd deZG dd deZdd ZdS ) z
Python parse tree definitions.
This is a very concrete parse tree; we need to keep every token and
even the comments and whitespace between tokens.
There's also a pattern matching implementation here.
z#Guido van Rossum N)StringIOic C sH t st| j jD ].\}}|| kr| j j | j j|= d| _ |S qW dS )z
Remove the node from the tree. Returns the position of the node in its
parent's children before it was removed.
N)r" enumerater# r& )r ir. r
r
r remove s
z Base.removec C sZ | j dkrdS xFt| j jD ]6\}}|| kr y| j j|d S tk
rP dS X q W dS )z
The node immediately following the invocant in their parent's children
list. If the invocant does not have a next sibling, it is None
Nr )r" r1 r#
IndexError)r r2 childr
r
r next_sibling s
zBase.next_siblingc C sP | j dkrdS x= 256), a sequence of
child nodes, and an optional context keyword argument.
As a side effect, the parent pointers of the children are updated.
N)r r! r# r" r; fixers_applied)r r r# contextr; rK r) r
r
r __init__ s
z
Node.__init__c C s d| j jt| j| jf S )z)Return a canonical string representation.z
%s(%s, %r))r rB r r r# )r r
r
r __repr__ s z
Node.__repr__c C s dj tt| jS )zk
Return a pretty string representation.
This reproduces the input source exactly.
r: )joinmapr? r# )r r
r
r __unicode__ s zNode.__unicode__r= r c C s | j | jf|j |jfkS )z Compare two nodes for equality.)r r# )r r r
r
r r s zNode._eqc C s t | jdd | jD | jdS )z$Return a cloned (deep) copy of self.c S s g | ] }|j qS r
)r ).0r) r
r
r
s z Node.clone..)rK )rJ r r# rK )r r
r
r r s z
Node.clonec c s( x | j D ]}|j E dH qW | V dS )z*Return a post-order iterator for the tree.N)r# r )r r5 r
r
r r s zNode.post_orderc c s( | V x | j D ]}|j E dH qW dS )z)Return a pre-order iterator for the tree.N)r# r )r r5 r
r
r r
s zNode.pre_orderc C s | j s
dS | j d jS )zO
The whitespace and comments preceding this node in the input.
r: r )r# r; )r r
r
r _prefix_getter s zNode._prefix_getterc C s | j r|| j d _d S )Nr )r# r; )r r; r
r
r _prefix_setter s zNode._prefix_setterc C s( | |_ d| j| _ || j|< | j dS )z
Equivalent to 'node.children[i] = child'. This method also sets the
child's parent attribute appropriately.
N)r" r# r& )r r2 r5 r
r
r set_child! s
zNode.set_childc C s | |_ | jj|| | j dS )z
Equivalent to 'node.children.insert(i, child)'. This method also sets
the child's parent attribute appropriately.
N)r" r# insertr& )r r2 r5 r
r
r insert_child+ s zNode.insert_childc C s | |_ | jj| | j dS )z
Equivalent to 'node.children.append(child)'. This method also sets the
child's parent attribute appropriately.
N)r" r# r% r& )r r5 r
r
r append_child4 s zNode.append_child)NNN)r= r )rB rC rD rE rM rN rQ rH rI rA r r r r rT rU rG r; rV rX rY r
r
r
r rJ s$
rJ c @ s e Zd ZdZdZdZdZddg fddZdd Zd d
Z e
j dkrFe Z d d
Z
dd Zdd Zdd Zdd Zdd Zdd ZeeeZdS )r, z'Concrete implementation for leaf nodes.r: r Nc C sF |dk r|\| _ \| _| _|| _|| _|dk r4|| _ |dd | _dS )z
Initializer.
Takes a type constant (a token number < 256), a string value, and an
optional context keyword argument.
N)_prefixr- columnr valuerK )r r r\ rL r; rK r
r
r rM G s z
Leaf.__init__c C s d| j j| j| jf S )z)Return a canonical string representation.z
%s(%r, %r))r rB r r\ )r r
r
r rN Z s z
Leaf.__repr__c C s | j t| j S )zk
Return a pretty string representation.
This reproduces the input source exactly.
)r; r? r\ )r r
r
r rQ ` s zLeaf.__unicode__r= c C s | j | jf|j |jfkS )z Compare two nodes for equality.)r r\ )r r r
r
r r k s zLeaf._eqc C s$ t | j| j| j| j| jff| jdS )z$Return a cloned (deep) copy of self.)rK )r, r r\ r; r- r[ rK )r r
r
r r o s
z
Leaf.clonec c s
| V d S )Nr
)r r
r
r r8 u s z Leaf.leavesc c s
| V dS )z*Return a post-order iterator for the tree.Nr
)r r
r
r r x s zLeaf.post_orderc c s
| V dS )z)Return a pre-order iterator for the tree.Nr
)r r
r
r r | s zLeaf.pre_orderc C s | j S )zP
The whitespace and comments preceding this token in the input.
)rZ )r r
r
r rT s zLeaf._prefix_getterc C s | j || _d S )N)r& rZ )r r; r
r
r rU s zLeaf._prefix_setter)r= r )rB rC rD rE rZ r- r[ rM rN rQ rH rI rA r r r8 r r rT rU rG r; r
r
r
r r, > s&
r, c C sN |\}}}}|s|| j krConstructor that prevents BasePattern from being instantiated.)r r )r r r r
r
r r s zBasePattern.__new__c C sL t | j| j| jg}x|r.|d d kr.|d= qW d| jjdjtt|f S )Nr z%s(%s)z, r` ) r r contentr r rB rO rP repr)r r r
r
r rN s
zBasePattern.__repr__c C s | S )z
A subclass can define this as a hook for optimizations.
Returns either self or another node with the same effect.
r
)r r
r
r optimize s zBasePattern.optimizec C sn | j dk r|j | j krdS | jdk rRd}|dk r4i }| j||sDdS |rR|j| |dk rj| jrj||| j< dS )a#
Does this pattern exactly match a node?
Returns True if it matches, False if not.
If results is not None, it must be a dict which will be
updated with the nodes matching named subpatterns.
Default implementation for non-wildcard patterns.
NFT)r ra _submatchupdater )r r. resultsrr
r
r match s
zBasePattern.matchc C s t |dkrdS | j|d |S )z
Does this pattern exactly match a sequence of nodes?
Default implementation for non-wildcard patterns.
r Fr )r] rh )r nodesrf r
r
r match_seq s zBasePattern.match_seqc c s& i }|r"| j |d |r"d|fV dS )z}
Generator yielding all matches for this pattern.
Default implementation for non-wildcard patterns.
r r N)rh )r ri rg r
r
r generate_matches s z BasePattern.generate_matches)N)N)
rB rC rD rE r ra r r rN rc rh rj rk r
r
r
r r_ s
r_ c @ s* e Zd ZdddZd ddZd
ddZdS ) LeafPatternNc C s& |dk r|dk r|| _ || _|| _dS )ap
Initializer. Takes optional type, content, and name.
The type, if given must be a token type (< 256). If not given,
this matches any *leaf* node; the content may still be required.
The content, if given, must be a string.
If a name is given, the matching node is stored in the results
dict under that key.
N)r ra r )r r ra r r
r
r rM s
zLeafPattern.__init__c C s t |tsdS tj| ||S )z*Override match() to insist on a leaf node.F)r r, r_ rh )r r. rf r
r
r rh s
zLeafPattern.matchc C s | j |jkS )a
Match the pattern's content to the node's children.
This assumes the node type matches and self.content is not None.
Returns True if it matches, False if not.
If results is not None, it must be a dict which will be
updated with the nodes matching named subpatterns.
When returning False, the results dict may still be updated.
)ra r\ )r r. rf r
r
r rd s
zLeafPattern._submatch)NNN)N)N)rB rC rD rM rh rd r
r
r
r rl s
rl c @ s$ e Zd ZdZdddZdddZdS ) NodePatternFNc C sT |dk r|dk r>t |}x$t|D ]\}}t|tr"d| _q"W || _|| _|| _dS )ad
Initializer. Takes optional type, content, and name.
The type, if given, must be a symbol type (>= 256). If the
type is None this matches *any* single node (leaf or not),
except if content is not None, in which it only matches
non-leaf nodes that also match the content pattern.
The content, if not None, must be a sequence of Patterns that
must match the node's children exactly. If the content is
given, the type must not be None.
If a name is given, the matching node is stored in the results
dict under that key.
NT)r! r1 r WildcardPattern wildcardsr ra r )r r ra r r2 itemr
r
r rM % s
zNodePattern.__init__c C s | j rJx>t| j|jD ],\}}|t|jkr|dk r>|j| dS qW dS t| jt|jkrbdS x*t| j|jD ]\}}|j||srdS qrW dS )a
Match the pattern's content to the node's children.
This assumes the node type matches and self.content is not None.
Returns True if it matches, False if not.
If results is not None, it must be a dict which will be
updated with the nodes matching named subpatterns.
When returning False, the results dict may still be updated.
NTF)ro rk ra r# r] re ziprh )r r. rf crg
subpatternr5 r
r
r rd B s
zNodePattern._submatch)NNN)N)rB rC rD ro rM rd r
r
r
r rm ! s
rm c @ s^ e Zd ZdZddedfddZdd Zddd Zdd
d Zd d
Z dd Z
dd Z dd Z dS )rn a
A wildcard pattern can match zero or more nodes.
This has all the flexibility needed to implement patterns like:
.* .+ .? .{m,n}
(a b c | d e | f)
(...)* (...)+ (...)? (...){m,n}
except it always uses non-greedy matching.
Nr c C s@ |dk r$t tt |}x |D ]}q W || _|| _|| _|| _dS )a
Initializer.
Args:
content: optional sequence of subsequences of patterns;
if absent, matches one node;
if present, each subsequence is an alternative [*]
min: optional minimum number of times to match, default 0
max: optional maximum number of times to match, default HUGE
name: optional name assigned to this match
[*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is
equivalent to (a b c | d e | f g h); if content is None,
this is equivalent to '.' in regular expression terms.
The min and max parameters work as follows:
min=0, max=maxint: .*
min=1, max=maxint: .+
min=0, max=1: .?
min=1, max=1: .
If content is not None, replace the dot with the parenthesized
list of alternatives, e.g. (a b c | d e | f g h)*
N)tuplerP ra minmaxr )r ra ru rv r altr
r
r rM l s
zWildcardPattern.__init__c C s d}| j dk r\}}|t|kr |dk rF|j| | jrFt||| j< dS q W dS )z4Does this pattern exactly match a sequence of nodes?NTF)rk r] re r r! )r ri rf rr rg r
r
r rj s
zWildcardPattern.match_seqc c s: | j dkrXxJt| jdtt|| j D ]*}i }| jrH|d| || j< ||fV q(W n| jdkrp| j|V nttdrtj }t
t_ zy@x:| j |dD ]*\}}| jr|d| || j< ||fV qW W nR t k
r x:| j
|D ],\}}| jr
|d| || j< ||fV qW Y nX W dttdr4|t_ X dS )a"
Generator yielding matches for a sequence of nodes.
Args:
nodes: sequence of nodes
Yields:
(count, results) tuples where:
count: the match comprises nodes[:count];
results: dict containing named submatches.
Nr Z bare_name getrefcountr )ra rangeru r] rv r _bare_name_matcheshasattrrH stderrr _recursive_matches RuntimeError_iterative_matches)r ri countrg Z save_stderrr
r
r rk s.
"
z WildcardPattern.generate_matchesc c s t |}d| jkr di fV g }x>| jD ]4}x.t||D ] \}}||fV |j||f q8W q(W x|rg }x|D ]\}} ||k rr|| jkrrxn| jD ]d}x^t|||d D ]H\}
} |
dkri }|j| |j| ||
|fV |j||
|f qW qW qrW |}qbW dS )z(Helper to iteratively yield the matches.r N)r] ru ra rk r% rv re ) r ri Znodelenrf rw rr rg Z new_resultsc0r0c1r1r
r
r r s*
z"WildcardPattern._iterative_matchesc C sx d}i }d}t |}xH| r\||k r\d}x0| jD ]&}|d j|| |r0|d7 }d}P q0W qW |d| || j< ||fS )z(Special optimized matcher for bare_name.r FTr N)r] ra rh r )r ri r rg donerv Zleafr
r
r rz s
z"WildcardPattern._bare_name_matchesc c s || j krdi fV || jk rxr| jD ]h}xbt||D ]T\}}xJ| j||d |d D ].\}}i }|j| |j| || |fV qXW q6W q&W dS )z(Helper to recursively yield the matches.r Nr )ru rv ra rk r} re ) r ri r rw r r r r rg r
r
r r} s
"
z"WildcardPattern._recursive_matches)N)N)
rB rC rD rE HUGErM rc rh rj rk r rz r} r
r
r
r rn ^ s #
- rn c @ s. e Zd Zd
ddZdd Zdd Zdd ZdS ) NegatedPatternNc C s |dk r|| _ dS )a
Initializer.
The argument is either a pattern or None. If it is None, this
only matches an empty sequence (effectively '$' in regex
lingo). If it is not None, this matches whenever the argument
pattern doesn't have any matches.
N)ra )r ra r
r
r rM s zNegatedPattern.__init__c C s dS )NFr
)r r. r
r
r rh ) s zNegatedPattern.matchc C s t |dkS )Nr )r] )r ri r
r
r rj - s zNegatedPattern.match_seqc c sL | j d kr"t|dkrHdi fV n&x| j j|D ]
\}}d S W di fV d S )Nr )ra r] rk )r ri rr rg r
r
r rk 1 s
z NegatedPattern.generate_matches)N)rB rC rD rM rh rj rk r
r
r
r r s
r c c s | sdi fV n| d | dd }}xl|j |D ]^\}}|sJ||fV q2xDt |||d D ].\}}i }|j| |j| || |fV q^W q2W dS )aR
Generator yielding matches for a sequence of patterns and nodes.
Args:
patterns: a sequence of patterns
nodes: a sequence of nodes
Yields:
(count, results) tuples where:
count: the entire sequence of patterns matches nodes[:count];
results: dict containing named submatches.
r r N)rk re ) Zpatternsri prestr r r r rg r
r
r rk = s
rk )rE
__author__rH warningsior r r r r r rJ r, r^ r_ rl rm rn r rk r
r
r
r s&
1nNV,= =#