Drop rows based on condition

Here are 2 ways to drop rows from a pandas data-frame based on a condition:

1. df = df[condition]

2. df.drop(df[condition].index, axis=0, inplace=True)

The first one does not do it inplace, right?

The second one does not work as expected when the index is not unique, so the user would need to reset_index() then set_index() back.

Question Would it be possible to have column dropping based directly on the condition? e.g. df.drop(condition, axis=0, inplace=True)

[Ashishjshetty]

you could probably look at DataFrame.where https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.where.html#pandas.DataFrame.where, it has inplace=True flag so it would work for you needs

example:

condition= df.a==1
df.where(cond=condition,inplace=True)

[zhuhuren]

Here’s my solution. (I can’t get the format of the code to work properly here.)

def drop_rows_by_condition(data, cond, temp_col=‘temp_col’): “”“When there are duplicates in index, dropping rows by condition is not straight forward. If 2 rows have the same index, one meets the condition and the other doesn’t, the 2 rows tend to be both dropped. This function solves this problem by using a temporary column. return: dropped rows “”” data_meet_criteria = data[cond] data[temp_col] = cond data.loc[cond, temp_col] = np.nan data.dropna(subset=[temp_col], inplace=True, axis=0) data.drop(temp_col, axis=1, inplace=True) return data_meet_criteria