.loc[...] = value returns SettingWithCopyWarning

Code Sample

# My code
df.loc[0, 'column_name'] = 'foo bar'

Problem description

This code in Pandas 20.3 throws SettingWithCopyWarning and suggests to

“Try using .loc[row_indexer,col_indexer] = value instead”.

I am already doing so, looks like there is a little bug. I use Jupyter. Thank you! 😃

Output of pd.show_versions()

---

commit: None python: 3.6.1.final.0 python-bits: 64 OS: Windows OS-release: 8.1 machine: AMD64 processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.20.1 pytest: 3.0.7 pip: 9.0.1 setuptools: 35.0.2 Cython: 0.25.2 numpy: 1.12.1 scipy: 0.19.0 xarray: None IPython: 5.3.0 sphinx: 1.5.6 patsy: 0.4.1 dateutil: 2.6.0 pytz: 2017.2 blosc: None bottleneck: 1.2.1 tables: 3.2.2 numexpr: 2.6.2 feather: None matplotlib: 2.0.2 openpyxl: None xlrd: 1.0.0 xlwt: 1.2.0 xlsxwriter: 0.9.6 lxml: 3.7.3 bs4: 4.6.0 html5lib: 0.999 sqlalchemy: 1.1.9 pymysql: None psycopg2: None jinja2: 2.9.6 s3fs: None pandas_gbq: None pandas_datareader: None

[CRiddler]

The issue here is that you’re slicing you dataframe first with .loc in line 4. The attempting to assign values to that slice.

df_c = df.loc[df.encountry == country, :]

Pandas isn’t 100% sure if you want to assign values to just your df_c slice, or have it propagate all the way back up to the original df. To avoid this when you first assign df_c make sure you tell pandas that it is its own data frame (and not a slice) by using

df_c = df.loc[df.encountry == country, :].copy()

Doing this will fix your error. I’ll tack on a brief example to help explain the above since I’ve noticed a lot of users get confused by pandas in this aspect.

Example with made up data

>>> import pandas as pd
>>> df = pd.DataFrame({'A':[1,2,3,4,5], 'B':list('QQQCC')})
>>> df
   A  B
0  1  Q
1  2  Q
2  3  Q
3  4  C
4  5  C
>>> df.loc[df['B'] == 'Q', 'new_col'] = 'hello'
>>> df
   A  B new_col
0  1  Q   hello
1  2  Q   hello
2  3  Q   hello
3  4  C     NaN
4  5  C     NaN

So the above works as we expect! Now lets try an example that mirrors what you attempted to do with your data.

>>> df = pd.DataFrame({'A':[1,2,3,4,5], 'B':list('QQQCC')})
>>> df_q = df.loc[df['B'] == 'Q']
>>> df_q
   A  B
0  1  Q
1  2  Q
2  3  Q
>>> df_q.loc[df['A'] < 3, 'new_col'] = 'hello'
/Users/riddellcd/anaconda/lib/python3.6/site-packages/pandas/core/indexing.py:337: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[key] = _infer_fill_value(value)

>>> df_q
   A  B new_col
0  1  Q   hello
1  2  Q   hello
2  3  Q     NaN

Looks like we hit the same error! But it changed df_q as we expected! This is because df_q is a slice of df so, even though we’re using .loc[] df_q pandas is warning us that it won’t propagate the changes up to df. To avoid this, we need to be more explicit and say that df_q is its own dataframe, separate from df by explicitly declaring it so.

Lets start back from df_q but use .copy() this time.

>>> df_q = df.loc[df['B'] == 'Q'].copy()
>>> df_q
   A  B
0  1  Q
1  2  Q
2  3  Q

Lets try to reassign our value now!
>>> df_q.loc[df['A'] < 3, 'new_col'] = 'hello'
>>> df_q
   A  B new_col
0  1  Q   hello
1  2  Q   hello
2  3  Q     NaN

This works without an error because we’ve told pandas that df_q is separate from df

If you in fact do want these changes to df_c to propagate up to df thats another point entirely and will answer if you want.

[TomAugspurger]

@NadiaRom Can you provide a full example? It’s hard to say for sure, but I suspect that df came from an operation that may be a view or copy. For example:

In [8]: df = pd.DataFrame({"A": [1, 2], "B": [3, 4], "C": [4, 5]})

In [9]: df1 = df[['A', 'B']]

In [10]: df1.loc[0, 'A'] = 5
/Users/taugspurger/Envs/pandas-dev/lib/python3.6/site-packages/pandas/pandas/core/indexing.py:180: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self._setitem_with_indexer(indexer, value)
/Users/taugspurger/Envs/pandas-dev/bin/ipython:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  #!/Users/taugspurger/Envs/pandas-dev/bin/python3.6

So we’re updating df1 correctly. The ambiguity is whether or not df will be updated as well. I think a similar thing is happening to you, but without a reproducible example it’s hard to say for sure.