question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[FEATURE-REQUEST] Vaex All Columns Dynamic Access Support

See original GitHub issue

Description I wish to aggregate all columns to a single column in vaex.

Something like :

df["combined"] = ",".join(df[reduced_cols])

Is your feature request related to a problem? Please describe. There is no simple way to do this in vaex.

Additional context Can be done in pandas using axis of apply, something like

df["combined"] = df[reduced_cols].apply(
    lambda row: ",".join(row.values.astype(str)), axis=1
)

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
JovanVeljanoskicommented, Aug 7, 2022

Hey,

Vaex in general does not support the axis argument i believe… so most if not all operations are column oriented (with exception of joins of course).

But there are relatively easy ways to accomplish what you are after. For example, first thing that comes to mind is:

import vaex

df = vaex.example()

# Get all the columns
columns = df.get_column_names()

# Build an expression in a loop
expr = df[columns[0]].astype('string')
for col in columns[1:]:
    expr += df[col].astype('string')

# Assign the expression to the dataframe
df['everything'] = expr

print(df)
1reaction
JovanVeljanoskicommented, Aug 29, 2022

Will close due to inactivity. Please reopen if needed

Read more comments on GitHub >

github_iconTop Results From Across the Web

Supported column datatypes, and whether stateful ML ...
What are the basic datatypes that vaex supports? ... and I'm passing the list of all columns from the dataframe dynamically, like:.
Read more >
Vaex: The process cannot access the file because it is being ...
I am working on an application that uses Vaex for accessing data from a feather file. We are creating virtual columns in a...
Read more >
Vaex introduction in 11 minutes
Central to Vaex is the DataFrame (similar, but more efficient than a Pandas ... A set of virtual columns, columns that are backed...
Read more >
The Garden of Forking Paths | Kaggle
Sometimes I fork the exact same kernel but time runs out ⓪ ou in a kernel, I suggest you click on "fork notebook"...
Read more >
Opinions on data scientists using Airflow : r/dataengineering
(E.g. "Run cell block 3 first then run 2 and 5 then run 4 between every iteration of 7-8). The most notable problem...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found