question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Clipping a data frame after selecting only numeric columns does not work

See original GitHub issue
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd
import numpy as np

df1 = pd.DataFrame({'A': [100] * 10, 'B': [100] * 10, 'C': ['abc'] * 10})
df1.loc[:, df1.select_dtypes(include=[np.number]).columns].clip(-90, 90, inplace=True)

Problem description

I am trying to clip the values of all numeric columns in the dataframe.

Initially, the data frame is

	A	B	C
0	100	100	abc
1	100	100	abc
2	100	100	abc
3	100	100	abc
4	100	100	abc
5	100	100	abc
6	100	100	abc
7	100	100	abc
8	100	100	abc
9	100	100	abc

The numeric columns are A and B, so df1.loc[:, df1.select_dtypes(include=[np.number]).columns] selects the required columns.

df1.loc[:, df1.select_dtypes(include=[np.number]).columns].clip(-90, 90, inplace=True) should clip values in A and B to 90.

Dataframe after clipping is:

	A	B	C
0	100	100	abc
1	100	100	abc
2	100	100	abc
3	100	100	abc
4	100	100	abc
5	100	100	abc
6	100	100	abc
7	100	100	abc
8	100	100	abc
9	100	100	abc

Clearly, the values are not getting clipped.

Expected Output

	A	B	C
0	90	90	abc
1	90	90	abc
2	90	90	abc
3	90	90	abc
4	90	90	abc
5	90	90	abc
6	90	90	abc
7	90	90	abc
8	90	90	abc
9	90	90	abc

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None

pandas : 1.0.3 numpy : 1.17.2 pytz : 2019.3 dateutil : 2.8.1 pip : 19.2.3 setuptools : 41.4.0 Cython : 0.29.13 pytest : 5.2.1 hypothesis : None sphinx : 2.4.4 blosc : None feather : None xlsxwriter : 1.2.1 lxml.etree : 4.4.1 html5lib : 1.0.1 pymysql : None psycopg2 : 2.8.4 (dt dec pq3 ext lo64) jinja2 : 2.10.3 IPython : 7.8.0 pandas_datareader: None bs4 : 4.8.0 bottleneck : 1.2.1 fastparquet : None gcsfs : None lxml.etree : 4.4.1 matplotlib : 3.1.1 numexpr : 2.7.0 odfpy : None openpyxl : 3.0.0 pandas_gbq : None pyarrow : None pytables : None pytest : 5.2.1 pyxlsb : None s3fs : None scipy : 1.3.1 sqlalchemy : 1.3.15 tables : 3.5.2 tabulate : 0.8.6 xarray : None xlrd : 1.2.0 xlwt : 1.3.0 xlsxwriter : 1.2.1 numba : 0.45.1

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
jrebackcommented, Apr 12, 2020

pls read the docs; https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html?highlight=settingwith#indexing-view-versus-copy

doing an inplace operation after indexing works in a copy of the data and this doesn’t do anything

this is generally but not always detectable (in this case it’s not)

0reactions
santhisenancommented, Apr 12, 2020

@dsaxton, shouldn’t SettingWithCopyWarning be thrown then? I think it is confusing otherwise.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Selecting only numeric columns from a data frame
Since a data frame is a list we can use the list-apply functions: nums <- unlist(lapply(x, is.numeric), use.names = FALSE). Then standard subsetting...
Read more >
Select Only Numeric Columns from Data Frame in R (Example)
How to extract only numeric and integer columns from a data frame in the ... select_if(data, is. numeric ) # Subset numeric columns...
Read more >
15 Easy Solutions To Your Data Frame Problems In R
Discover how to create a data frame in R, change column and row names, access values, attach data frames, apply functions and much...
Read more >
Data filtering in Pandas - Towards Data Science
The complete guide to clean data sets — Part 3 ... Filtering data from a data frame is one of the most common...
Read more >
Fix text-formatted numbers by applying a number format
Numbers that are formatted as text can cause problems with calculations or ... If the row or column contains data, Ctrl+Shift+Arrow key selects...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found