scope issues in comprehensions

Example JSON

{
   "foods": [
      { "name": "carrot" },
      { "name": "banana" }
   ],
   "people": [
      { "name": "alice", "likes": "apples" },
      { "name": "bob", "likes": "banana" },
      { "name": "carrol", "likes": "carrot" },
      { "name": "dave", "likes": "donuts" }
   ]
}

What happens as of 99f6771

$ jello "foods = set(f.name for f in _.foods)
         [p.name for p in _.people if p.likes not in foods]"
jello:  Query Exception:  NameError
        name 'foods' is not defined
        query:  foods = set(f.name for f in _.foods)\n[p.name for p in _.people if p.likes not in foods]
        data:  {'foods': [{'name': 'carrot'}, {'name': 'banana'}], 'people': [{'name': 'alice', 'likes': 'apples'}, {'name': 'bob', 'likes': 'banana'}, {'name': 'carrol', 'likes': 'carrot'}, {'name':
            'dave', 'likes': 'donuts'}]}

$ jello "[p.name for p in _.people if p.likes not in (f.name for f in _.foods)]"
jello:  Query Exception:  NameError
        name '_' is not defined
        query:  [p.name for p in _.people if p.likes not in (f.name for f in _.foods)]
        data:  {'foods': [{'name': 'carrot'}, {'name': 'banana'}], 'people': [{'name': 'alice', 'likes': 'apples'}, {'name': 'bob', 'likes': 'banana'}, {'name': 'carrol', 'likes': 'carrot'}, {'name':
            'dave', 'likes': 'donuts'}]}

What I expected to happen

$ jello "foods = set(f.name for f in _.foods)
         [p.name for p in _.people if p.likes not in foods]"
[
  "alice",
  "dave"
]

$ jello "[p.name for p in _.people if p.likes not in (f.name for f in _.foods)]"
[
  "alice",
  "dave"
]

Suggested change

diff --git a/jello/lib.py b/jello/lib.py
index 60b442c..ed2f4cc 100644
--- a/jello/lib.py
+++ b/jello/lib.py
@@ -459,10 +459,12 @@ def pyquery(_θ_data, _θ_query):
     del _θ_query
     _θ_last = ast.Expression(_θ_block.body.pop().value)    # assumes last node is an expression
 
-    exec(compile(_θ_block, '<string>', mode='exec'))
+    _θ_scope = {'_': _}
+
+    exec(compile(_θ_block, '<string>', mode='exec'), _θ_scope)
     del _θ_block
 
-    _θ_output = eval(compile(_θ_last, '<string>', mode='eval'))
+    _θ_output = eval(compile(_θ_last, '<string>', mode='eval'), _θ_scope)
     del _θ_last
 
     # convert output back to normal dict

[kellyjonbrazil]

Nice - thanks! Yes, that was a struggle to find out how to do get that to work similar to a REPL. I’ll definitely consider the option you describe.

Yes, I was using the strange variable names and del to keep from polluting the scope too much, but also wanted some convenience modules pre-imported like os.environ. I’ll dig into this a bit more. Thanks again!

[sqwishy]

oh one real quick bonus meme because you mentioned this

I feel like I have run into other oddities in the past and this might be the fix for those as well.

so I noticed you remove the last statement from the block and eval it separately in order to get the result of the expression. alternatively, if the last statement in the block is an expression you could leave it in but change it to an assignment statement so it stores to some variable ‘__final’ or something and that will be written to the scope dictionary passed in to exec when the block is exectued. this way there is no need for the eval just the exec on the block

But I don’t actually know if this is better or not. I couldn’t find a situation where this actually improved anything so it maybe doesn’t. Just an idea if the current approach doesn’t work for some reason in the future or whatever.