Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Inconsistent replacement of values

See original GitHub issue

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Problem description

Value at a given position don’t change using “values” indexing when the column in cuestion has object data type. However, the value change using “at”.

I known that values return numpy array, then I test the same behaviour on numpy and everything is ok.

Normal Behaviour:

import pandas as pd

#Create DataFrame
columns = ["A", "B"]

data = [[0, 1], [2, 3]]

dataset = pd.DataFrame(data=data, columns=columns)

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)
print("="*25)

#Modify value
dataset.values[0, 1] = "4"

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)
print("="*25)

#Modify value
dataset.values[0, 1] = 4

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)

Output:

   A  B
0  0  1
1  2  3

 A    int64
B    int64
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
   A  B
0  0  4
1  2  3

 A    int64
B    int64
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
   A  B
0  0  4
1  2  3

 A    int64
B    int64
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

Inconsistent Behaviour:

import pandas as pd

#Create DataFrame
columns = ["A", "B"]

data = [[0, 1], [2, "3"]]

dataset = pd.DataFrame(data=data, columns=columns)

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)
print("="*25)

#Modify value
dataset.values[0, 1] = "4"

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)
print("="*25)

#Modify value
dataset.values[0, 1] = 4

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)
print("="*25)

#Modify value
dataset.at[0, "B"] = "4"

print(dataset)
print("\n", dataset.dtypes)
print("\n", dataset.values.flags)

Output:

   A  B
0  0  1
1  2  3

 A     int64
B    object
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
   A  B
0  0  1
1  2  3

 A     int64
B    object
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
   A  B
0  0  1
1  2  3

 A     int64
B    object
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
   A  B
0  0  4
1  2  3

 A     int64
B    object
dtype: object

   C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

Normal Behaviour on Numpy:

import numpy as np

data = [[0, 1], [2, "3"]]

#Create array
dataset = np.array(data)

print(dataset)
print("\n", dataset.dtype)
print("\n", dataset.flags)
print("="*25)

#Modify value
dataset[0, 1] = 4

print(dataset)
print("\n", dataset.dtype)
print("\n", dataset.flags)
print("="*25)

#Create array
dataset = np.array(data, dtype=object)

print(dataset)
print("\n", dataset.dtype)
print("\n", dataset.flags)
print("="*25)

#Modify value
dataset[0, 1] = 4

print(dataset)
print("\n", dataset.dtype)
print("\n", dataset.flags)

Output:

[['0' '1']
 ['2' '3']]

 <U21

   C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
[['0' '4']
 ['2' '3']]

 <U21

   C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
[[0 1]
 [2 '3']]

 object

   C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

=========================
[[0 4]
 [2 '3']]

 object

   C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

Output of pd.__version__

'1.0.3'

Output of pd.show_versions()

/usr/local/lib/python3.6/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/usr/local/lib/python3.6/dist-packages/pandas_datareader/compat/__init__.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.9.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.19.104+
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.0.3
numpy            : 1.18.2
pytz             : 2018.9
dateutil         : 2.8.1
pip              : 19.3.1
setuptools       : 46.0.0
Cython           : 0.29.16
pytest           : 3.6.4
hypothesis       : None
sphinx           : 1.8.5
blosc            : None
feather          : 0.4.0
xlsxwriter       : None
lxml.etree       : 4.2.6
html5lib         : 1.0.1
pymysql          : None
psycopg2         : 2.7.6.1 (dt dec pq3 ext lo64)
jinja2           : 2.11.1
IPython          : 5.5.0
pandas_datareader: 0.8.1
bs4              : 4.6.3
bottleneck       : 1.3.2
fastparquet      : None
gcsfs            : None
lxml.etree       : 4.2.6
matplotlib       : 3.2.1
numexpr          : 2.7.1
odfpy            : None
openpyxl         : 2.5.9
pandas_gbq       : 0.11.0
pyarrow          : 0.14.1
pytables         : None
pytest           : 3.6.4
pyxlsb           : None
s3fs             : 0.4.2
scipy            : 1.4.1
sqlalchemy       : 1.3.15
tables           : 3.4.4
tabulate         : 0.8.7
xarray           : 0.15.1
xlrd             : 1.1.0
xlwt             : 1.3.0
xlsxwriter       : None
numba            : 0.47.0

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
jbrockmendelcommented, Apr 7, 2020

In that case, the recommendation would be use iloc instead values and avoid it unless you want to display some values or don’t use values for anything?

use .iloc for setting, yes. .values really should only be used if you need to pass a numpy array to something outside of pandas

0reactions
jbrockmendelcommented, Apr 7, 2020

Closing since this isnt a bug, but happy to discuss this further.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Color Replacement Seems inconsistent - Pixelmator Community
The bug is related to what I perceive to be an inconsistent application of color replacement. I created a simple black to white...
Read more >
Bug Patterns - Error Prone
The contract of Object.equals() states that for any non-null reference value x, x.equals(null) should return false. If x is null, a NullPointerException ...
Read more >
[Koha-bugs] [Bug 7045] Default-value substitution inconsistent
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7045. Katrin Fischer <katrin.fisc...@bsz-bw.de> changed: What |Removed |Added ...
Read more >
c++ - std::is_constructible returns inconsistent value for private ...
When the trait is checked again in the next line or possibly at a completely different location (as is the case here), it...
Read more >
Inconsistent (incorrect?) behavior of collapse with extended ...
... all values in a group are missing: clear set obs 3 gen x = . replace x ... Mauricio found a bug...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

BLD: Compatibility issues with master branch cython

Next page

BUG: User-facing AssertionError in Series.to_timestamp