pure_eval
This is a Python package that lets you safely evaluate certain AST nodes without triggering arbitrary code that may have unwanted side effects.
It can be installed from PyPI:
pip install pure_eval
To demonstrate usage, suppose we have an object defined as follows:
class Rectangle:
def init(self, width, height):
self.width = width
self.height = height
@property
def area(self):
print("Calculating area...")
return self.width * self.height
rect = Rectangle(3, 5)
Given the rect object, we want to evaluate whatever expressions we can in this source code:
source = "(rect.width, rect.height, rect.area)"
This library works with the AST, so let's parse the source code and peek inside:
import ast
tree = ast.parse(source)
thetuple = tree.body[0].value
for node in thetuple.elts:
print(ast.dump(node))
Output:
Attribute(value=Name(id='rect', ctx=Load()), attr='width', ctx=Load())
Attribute(value=Name(id='rect', ctx=Load()), attr='height', ctx=Load())
Attribute(value=Name(id='rect', ctx=Load()), attr='area', ctx=Load())
Now to actually use the library. First construct an Evaluator:
from pure_eval import Evaluator
evaluator = Evaluator({"rect": rect})
The argument to Evaluator should be a mapping from variable names to their values. Or if you have access to the stack frame where rect is defined, you can instead use:
evaluator = Evaluator.from_frame(frame)
Now to evaluate some nodes, using evaluator[node]:
print("rect.width:", evaluator[thetuple.elts[0]])
print("rect:", evaluator[thetuple.elts[0].value])
Output:
rect.width: 3
rect: <__main__.Rectangle object at 0x105b0dd30>
OK, but you could have done the same thing with eval. The useful part is that it will refuse to evaluate the property rect.area because that would trigger unknown code. If we try, it'll raise a CannotEval exception.
from pure_eval import CannotEval
try:
print("rect.area:", evaluator[the_tuple.elts[2]]) # fails
except CannotEval as e:
print(e) # prints CannotEval
To find all the expressions that can be evaluated in a tree:
for node, value in evaluator.find_expressions(tree):
print(ast.dump(node), value)
Output:
Attribute(value=Name(id='rect', ctx=Load()), attr='width', ctx=Load()) 3
Attribute(value=Name(id='rect', ctx=Load()), attr='height', ctx=Load()) 5
Name(id='rect', ctx=Load()) <main.Rectangle object at 0x105568d30>
Name(id='rect', ctx=Load()) <main.Rectangle object at 0x105568d30>
Name(id='rect', ctx=Load()) <main.Rectangle object at 0x105568d30>
Note that this includes rect three times, once for each appearance in the source code. Since all these nodes are equivalent, we can group them together:
from pureeval import groupexpressions
for nodes, values in groupexpressions(evaluator.findexpressions(tree)):
print(len(nodes), "nodes with value:", values)
Output:
1 nodes with value: 3
1 nodes with value: 5
3 nodes with value: <__main__.Rectangle object at 0x10d374d30>
If we want to list all the expressions in a tree, we may want to filter out certain expressions whose values are obvious. For example, suppose we have a function `foo`:
<div class="codehilite">
<pre><span></span><code><span class="k">def</span> <span class="nf">foo</span><span class="p">():</span>
<span class="k">pass</span>
</code></pre>
</div>
If we refer to `foo` by its name as usual, then that's not interesting:
<div class="codehilite">
<pre><span></span><code><span class="kn">from</span> <span class="nn">pure_eval</span> <span class="kn">import</span> <span class="n">is_expression_interesting</span>
<span class="n">node</span> <span class="o">=</span> <span class="n">ast</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)</span><span class="o">.</span><span class="n">body</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">value</span>
<span class="nb">print</span><span class="p">(</span><span class="n">ast</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">node</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">is_expression_interesting</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">foo</span><span class="p">))</span>
</code></pre>
</div>
Output:
<div class="codehilite">
<pre><span></span><code><span class="n">Name</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s1">'foo'</span><span class="p">,</span> <span class="n">ctx</span><span class="o">=</span><span class="n">Load</span><span class="p">())</span>
<span class="kc">False</span>
</code></pre>
</div>
But if we refer to it by a different name, then it's interesting:
<div class="codehilite">
<pre><span></span><code><span class="n">node</span> <span class="o">=</span> <span class="n">ast</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s1">'bar'</span><span class="p">)</span><span class="o">.</span><span class="n">body</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">value</span>
<span class="nb">print</span><span class="p">(</span><span class="n">ast</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">node</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">is_expression_interesting</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">foo</span><span class="p">))</span>
</code></pre>
</div>
Output:
<div class="codehilite">
<pre><span></span><code><span class="n">Name</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s1">'bar'</span><span class="p">,</span> <span class="n">ctx</span><span class="o">=</span><span class="n">Load</span><span class="p">())</span>
<span class="kc">True</span>
</code></pre>
</div>
In general `is_expression_interesting` returns False for the following values:
- Literals (e.g. `123`, `'abc'`, `[1, 2, 3]`, `{'a': (), 'b': ([1, 2], [3])}`)
- Variables or attributes whose name is equal to the value's `__name__`, such as `foo` above or `self.foo` if it was a method.
- Builtins (e.g. `len`) referred to by their usual name.
To make things easier, you can combine finding expressions, grouping them, and filtering out the obvious ones with:
<div class="codehilite">
<pre><span></span><code><span class="n">evaluator</span><span class="o">.</span><span class="n">interesting_expressions_grouped</span><span class="p">(</span><span class="n">root</span><span class="p">)</span>
</code></pre>
</div>
To get the source code of an AST node, I recommend [asttokens](https://github.com/gristlabs/asttokens).
Here's a complete example that brings it all together:
<div class="codehilite">
<pre><span></span><code><span class="kn">from</span> <span class="nn">asttokens</span> <span class="kn">import</span> <span class="n">ASTTokens</span>
<span class="kn">from</span> <span class="nn">pure_eval</span> <span class="kn">import</span> <span class="n">Evaluator</span>
<span class="n">source</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2">x = 1</span>
<span class="s2">d = </span><span class="si">{x: 2}</span>
<span class="s2">y = d[x]</span>
<span class="s2">"""</span>
<span class="n">names</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">exec</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">names</span><span class="p">)</span>
<span class="n">atok</span> <span class="o">=</span> <span class="n">ASTTokens</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">parse</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">for</span> <span class="n">nodes</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">Evaluator</span><span class="p">(</span><span class="n">names</span><span class="p">)</span><span class="o">.</span><span class="n">interesting_expressions_grouped</span><span class="p">(</span><span class="n">atok</span><span class="o">.</span><span class="n">tree</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">atok</span><span class="o">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">nodes</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="s2">"="</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
</code></pre>
</div>
Output:
<div class="codehilite">
<pre><span></span><code><span class="n">x</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span><span class="mi">1</span><span class="p">:</span> <span class="mi">2</span><span class="p">}</span>
<span class="n">y</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">d</span><span class="p">[</span><span class="n">x</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
</code></pre>
</div>