BUG: Clipping a data frame after selecting only numeric columns does not work
See original GitHub issue-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
(optional) I have confirmed this bug exists on the master branch of pandas.
Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Code Sample, a copy-pastable example
import pandas as pd
import numpy as np
df1 = pd.DataFrame({'A': [100] * 10, 'B': [100] * 10, 'C': ['abc'] * 10})
df1.loc[:, df1.select_dtypes(include=[np.number]).columns].clip(-90, 90, inplace=True)
Problem description
I am trying to clip the values of all numeric columns in the dataframe.
Initially, the data frame is
A B C
0 100 100 abc
1 100 100 abc
2 100 100 abc
3 100 100 abc
4 100 100 abc
5 100 100 abc
6 100 100 abc
7 100 100 abc
8 100 100 abc
9 100 100 abc
The numeric columns are A and B, so
df1.loc[:, df1.select_dtypes(include=[np.number]).columns]
selects the required columns.
df1.loc[:, df1.select_dtypes(include=[np.number]).columns].clip(-90, 90, inplace=True)
should clip values in A and B to 90.
Dataframe after clipping is:
A B C
0 100 100 abc
1 100 100 abc
2 100 100 abc
3 100 100 abc
4 100 100 abc
5 100 100 abc
6 100 100 abc
7 100 100 abc
8 100 100 abc
9 100 100 abc
Clearly, the values are not getting clipped.
Expected Output
A B C
0 90 90 abc
1 90 90 abc
2 90 90 abc
3 90 90 abc
4 90 90 abc
5 90 90 abc
6 90 90 abc
7 90 90 abc
8 90 90 abc
9 90 90 abc
Output of pd.show_versions()
INSTALLED VERSIONS
commit : None
pandas : 1.0.3 numpy : 1.17.2 pytz : 2019.3 dateutil : 2.8.1 pip : 19.2.3 setuptools : 41.4.0 Cython : 0.29.13 pytest : 5.2.1 hypothesis : None sphinx : 2.4.4 blosc : None feather : None xlsxwriter : 1.2.1 lxml.etree : 4.4.1 html5lib : 1.0.1 pymysql : None psycopg2 : 2.8.4 (dt dec pq3 ext lo64) jinja2 : 2.10.3 IPython : 7.8.0 pandas_datareader: None bs4 : 4.8.0 bottleneck : 1.2.1 fastparquet : None gcsfs : None lxml.etree : 4.4.1 matplotlib : 3.1.1 numexpr : 2.7.0 odfpy : None openpyxl : 3.0.0 pandas_gbq : None pyarrow : None pytables : None pytest : 5.2.1 pyxlsb : None s3fs : None scipy : 1.3.1 sqlalchemy : 1.3.15 tables : 3.5.2 tabulate : 0.8.6 xarray : None xlrd : 1.2.0 xlwt : 1.3.0 xlsxwriter : 1.2.1 numba : 0.45.1
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (2 by maintainers)
Top GitHub Comments
pls read the docs; https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html?highlight=settingwith#indexing-view-versus-copy
doing an inplace operation after indexing works in a copy of the data and this doesn’t do anything
this is generally but not always detectable (in this case it’s not)
@dsaxton, shouldn’t SettingWithCopyWarning be thrown then? I think it is confusing otherwise.