[OP] Add apply lambda op to use with UDFs

Is your feature request related to a problem? Please describe.

df.apply and df.apply(lambda: x… ) are two methods commonly used to apply UDFs in W&D model preprocessing and feature engineering.

Describe the solution you’d like We should be able to create UDFs and use with df.apply or df.apply(lambda: x...)

Additional context cudf does not have apply or apply lambda functionality yet. There is applymap method that applies an elementwise function to transform the values in the Column.

Pandas example:

pdf = pd.DataFrame({'display_id':['1', '2','3', '4'], 'clicks':[0, 1, 1, 0], 'views':[1, 5,10, 3]})
pdf.loc[:, 'ctr'] = pdf.apply(ctr_udf, axis=1)
pdf.head()

     display_id  clicks  views  ctr
0	1	0	1	0.0
1	2	1	5	0.2
2	3	1	10	0.1
3	4	0	3	0.0

[bschifferer]

No, I think you can apply it like:

f = lambda gdf: gdf['clicks']/gdf['views']
gdf['ctr'] = f(gdf)

[benfred]

I don’t think we should be letting people apply python functions at the row level. Can we restrict this to operations happening at the dataframe or series level instead? If we are calling a python function per row we won’t be able to have acceptable performance (which is why there isn’t an ‘apply’ function in cudf right now).

For your example of calculating CTR, we could add something where the calculation per GDF chunk looks like:

gdf['ctr'] = gdf['views'] / gdf['displays']

instead of using an apply function. The idea here is that we are declaring the work that needs done in python, but the actual work is done on the gpu using cudf. (also take a look https://github.com/NVIDIA/NVTabular/pull/84#discussion_r438953563 for how to express a similar idea as a NVT operator).