[BUG-REPORT] to_pandas_df throws exception on filtered data frame on virtual column, with empty result

Description to_pandas_df throws exception when invoked on empty filter. This happens only when there is a virtual column with a transformation (it will not happen if col1 is not modified see code below)

Software information

• Vaex version (import vaex; vaex.__version__): {‘vaex’: ‘4.12.0’, ‘vaex-core’: ‘4.12.0’, ‘vaex-viz’: ‘0.5.3’, ‘vaex-hdf5’: ‘0.12.3’, ‘vaex-server’: ‘0.8.1’, ‘vaex-astro’: ‘0.9.1’, ‘vaex-jupyter’: ‘0.8.0’, ‘vaex-ml’: ‘0.18.0’}
• Vaex was installed via: pip
• OS: Arch (similar behavior on Ubuntu)

Additional information Code to reproduce. First two prints work, third one throws exception.

import pandas as pd
import vaex as vx

df = vx.from_pandas(pd.DataFrame(data={'col1':['chr1','chr2'],'col2':[3,4]}))
print(df[df['col1']=='3'].extract().to_pandas_df()) # Works
df['col1'] = df.col1.astype('str').str.replace('^chr','',regex=True)
print(df[df['col1']=='3'].extract()) # Works
print(df[df['col1']=='3'].extract().to_pandas_df()) # Throws exception

results in: AssertionError for assert filter.sum() == expected_length, in dataset.py, line 1018, in slice

[maartenbreddels]

Ok, so, with an extract that dataframe is a bit in limbo, some things can work, depending if you use the virtual column or not. I think #2284 is the better way for now.

[JovanVeljanoski]

Did it work in the case of virtual columns (that are not materialized etc.). I don’t really see how it could have worked… since we do not know the dtype before evaluation (if I am not mistaken)