question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

New behaviour regarding inplace values setting with iloc

See original GitHub issue

In scikit-learn, when testing the pandas nightly build, we got a FutureWarning related to the following deprecation:

https://pandas.pydata.org/docs/dev/whatsnew/v1.5.0.html#try-operating-inplace-when-setting-values-with-loc-and-iloc

We have 2 related questions regarding this deprecation. First, it seems that we cannot reproduce the “Old Behaviour” with the latest available release:

In [1]: import pandas as pd

In [2]: import numpy as np

In [3]: pd.__version__
Out[3]: '1.4.2'

In [4]: values = np.arange(4).reshape(2, 2)
   ...: 
   ...: df = pd.DataFrame(values)
   ...: 
   ...: ser = df[0]

In [5]: df.iloc[:, 0] = np.array([10, 11])

In [6]: ser
Out[6]: 
0    10
1    11
Name: 0, dtype: int64

Is there a reason for not spotting the behaviour shown in the documentation?

The second question (actually it is more a comment to open a discussion) concerns the proposed fix.

It is proposed to use df[df.columns[i]] = newvals instead of the df.iloc[:, i] = newvals. I personally find this way a bit counterintuitive since the SettingWithCopyWarning proposes to change to df.loc[rows, cols] instead of df[cols][rows] to get the inplace behaviour.

If we consider that both approaches intend for an inplace change, the patterns used for “by position” (i.e. .iloc) or “by label” (i.e. .loc) are really different.

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:1
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
jorisvandenbosschecommented, Jun 22, 2022

It seems like when using .iloc on an empty dataframe, you get a similar warning which could maybe be avoided?

Indeed, in this case, there is an “enlargement” of the dataframe when setting the values, and that can never be done inplace, and so we should avoid the warning in that case.

0reactions
glemaitrecommented, Jul 20, 2022

Indeed.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to get rid of the in place FutureWarning when setting an ...
The FutureWarning informs about a planned semantic change, when using iloc : the change will be done in-place in future versions.
Read more >
25. Accessing and Changing values of DataFrames
This chapter of our Pandas and Python tutorial will show various ways to access and change selectively values in Pandas DataFrames and Series....
Read more >
Indexing and selecting data — pandas 1.5.2 documentation
Different choices for indexing# ·.loc is primarily label based, but may also be used with a boolean array. . ·.iloc is primarily integer...
Read more >
Views and Copies in pandas - Practical Data Science
Unfortunately, while this behavior is relatively straightforward in numpy, ... SettingWithCopyWarning: A value is trying to be set on a copy of a...
Read more >
How to change or update a specific cell in Python Pandas ...
iat - Access a single value for a row/column pair by integer position. Use iat if you only need to get or set...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found