Allow columns to "pass through" summarize? (e.g. across)

I have another question which can really optimize the way I work: often I’m performing calculations on aggregates and would like to allow some features (that are constant within the group) to pass through after the summarize. I know it’s possible to create new variables in the sense of summarize(new_col=_.feature.mean(), old_col=_.old_col.iloc[0]), for example, but this gets tedious if there are many columns (or even with a few columns).

Is there a way to tell siuba (more specifically summarize) to pass through some variables? And on a related note - is there a way to make the same operation on many columns without having to use gather? (Currently I have the process of gather -> group_by -> summarize -> spread to operate on many same columns)?

Thanks for the awesome library! Omri

[machow]

Hey, sorry for the delay–I’ve been thinking about how across could be implemented. It seems like, similar to siuba’s implementation of case_when(), across() could essentially take data as its first argument (verbs do this too. e.g. select or mutate).

Here’s a case_when example (since apparently it is undocumented 😬).

from siuba.data import mtcars
from siuba import case_when

# outputs numpy array: array(['> 4 cyl', '> 4 cyl', 'other', ...]))
case_when(mtcars, {_.cyl > 4: "> 4 cyl", True: "other"})

# outputs a Symbolic expression
case_when(_, {}) 

# note that case_when works in SQL backends too!

Across proposal

Essentially what could happen is:

1. across takes data as its first argument (e.g. across(_, ...), across(mtcars, ...))

• its other args are like dplyr: column selection, functions to apply, etc…
• it returns a DataFrame(GroupBy)

1. across(_, _.contains('abc'), _.mean(), ...) within verbs will just get evaluated like other symbolic calls

• down the road we can probably omit the first _

1. functions like mutate and summarize will need to be able to handle when evaluated arguments are DataFrame(GroupBy). This would be super handy to add anyway!

Examples

from siuba.data import mtcars
from siuba import _, across

# all the classic selection options
across(mtcars, [_.mpg, _.hp], _.mean())
across(mtcars, _.contains("mpg"), _.mean())
across(_, _[3:5], _.mean())

# with summarize
mtcars >> summarize(across(_, [_.mpg, _.hp], _.mean()))

# an implication of summarize accepting DataFrames as arguments is
# that you can do this.
mtcars >> summarize(_[["mpg", "hp"]].mean())

# or this, which would override mpg and hp to be + 1
mtcars >> mutate(_[["mpg", "hp"]] + 1)

# here is analogous dplyr code. TODO, is the overriding behavior mentioned
# explicitly in dplyr docs?
mutate(mtcars, mtcars[c("hp", "mpg")] + 1)

[moeketsims]

does siuba have across verb?