ENH: remove restrictions to numexpr to allow `where` etc.

Is your feature request related to a problem?

the evaluation of a query is currently limited to the list _mathops while numexpr would support more, most notably a where (that would also solve other issues simple).

I do not see any reason for this restriction. In fact, simply adding the where runs (at least for my use case). Why is this restriction in place? Why can’t we enlarge it/directly pass it through to numexpr?

Describe the solution you’d like

Allow the full operator set that numexpr supports in the pd.eval

API breaking implications

Nothing

Alternatives

Using .where is an option if you can access the dataframe directly (although suboptimal). However, if your selection of data is based on passing a selection string around instead of the df (several reasons for this), the latter is not feasible.

The following doesn’t work:

import pandas as pd

data = {'a': [1, 2, 3]}                                                 

df = pd.DataFrame({'a': [1, 2, 3]})
df.eval('where(a>2, 42, 0)')

whereas in numexpr it does

numexpr.evaluate('where(a>2, 42, 0)', local_dict=data)

we expect this to return [0, 0, 42]

[achimgaedke]

I have “solved” my problem by using

import pandas
pandas.core.computation.ops.MATHOPS = (*pandas.core.computation.ops.MATHOPS, "where")

Works out of the box with ternary operators, 🎉

df = pandas.DataFrame({"a": [2.0, 4.0, 5.0]})
pandas.eval("where(df.a > 3.0, df.a, 1)", target=df)

results in

0    1.0
1    4.0
2    5.0
Name: done, dtype: float64

Not exactly proud of this solution, but this shows that this feature request is probably done by adding the strings and writing some unit tests.

NB: arguments 2 and 3 have to be numbers - I’d love to have strings (same type should only be required only for arg 2 and 3) NB2: (df.a>3.0) * df.a + (df.a<=3) * 1 works like where(df.a > 3.0, df.a, 1)

[jonas-eschle]

Oc, sorry that was by mistake and it is not yet resolved.

@TomAugspurger, do you have an idea why this is not here?