Spyder debugger breaks when using pandas MultiIndex

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Appending a column to the index works fine while running, but fails when in debug mode. I’m using Spyder 5.3.0 on Windows with pandas 1.4.2.

I’ve created some dummy code that shows the problem:

import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3], "b": [100, 200, 300], "c": ["a", "b", "c"]})

df.set_index("a", inplace=True)
df.set_index("b", append=True, inplace=True)

print(df)
print(df.index)

I thought I would work around it with:

# df.set_index("b", append=True, inplace=True)
df = df.reset_index().set_index(["a", "b"])

But the same issue persists.

Running this using pdb does work as expected

As it is a pandas error I’ve added this also to their issue tracker: https://github.com/pandas-dev/pandas/issues/46890

What steps reproduce the problem?

1. Create a script that adds a MultiIndex in a pandas DataFrame (for example, see above)
2. Run it in Spyder debugging mode
3. Step through the code, over the set_index() line.

What is the expected output? What do you see instead?

Expected output:

       c
a b     
1 100  a
2 200  b
3 300  c
MultiIndex([(1, 100),
            (2, 200),
            (3, 300)],
           names=['a', 'b'])

Actual output:

Traceback (most recent call last):
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\spyder_kernels\customize\spyderpdb.py", line 776, in run
    super(SpyderPdb, self).run(cmd, globals, locals)
  File "C:\Users\username\Miniconda3\envs\some-env\lib\bdb.py", line 597, in run
    exec(cmd, globals, locals)
  File "c:\users\username\path\temp.py", line 6, in <module>
    df.set_index("b", append=True, inplace=True)
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\pandas\util\_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\pandas\core\frame.py", line 5560, in set_index
    index._cleanup()
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\pandas\core\indexes\base.py", line 843, in _cleanup
    self._engine.clear_mapping()
  File "pandas\_libs\properties.pyx", line 37, in pandas._libs.properties.CachedProperty.__get__
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\pandas\core\indexes\multi.py", line 1097, in _engine
    return MultiIndexUIntEngine(self.levels, self.codes, offsets)
  File "pandas\_libs\index.pyx", line 635, in pandas._libs.index.BaseMultiIndexCodesEngine.__init__
  File "C:\Users\username\Miniconda3\envs\some-env\lib\site-packages\pandas\core\indexes\multi.py", line 136, in _codes_to_ints
    codes <<= self.offsets
AttributeError: 'MultiIndex' object has no attribute 'offsets'

Paste Traceback/Error Below (if applicable)

N/A

Versions

• Spyder version: 5.3.0 None
• Python version: 3.8.10 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.6
• Operating System: Windows 10

Dependencies

Fold out for dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree                  :  None (OK)
IPython >=7.31.1;<8.0.0       :  7.32.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.4.0 (OK)
keyring >=17.0.0              :  23.5.0 (OK)
nbconvert >=4.0               :  6.4.5 (OK)
numpydoc >=0.6.0              :  1.2.1 (OK)
paramiko >=2.4.0              :  2.10.3 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.11.2 (OK)
pylint >=2.5.0                :  2.13.3 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0          :  1.4.1 (OK)
pylsp_black >=1.2.0           :  1.2.0 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0      :  5.3.0 (OK)
qtpy >=2.0.1                  :  2.0.1 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  61.2.0 (OK)
sphinx >=0.6.6                :  4.5.0 (OK)
spyder_kernels >=2.3.0;<2.4.0 :  2.3.0 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog                      :  2.1.7 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  0.29.28 (OK)
matplotlib >=3.0.0            :  3.5.1 (OK)
numpy >=1.7                   :  1.22.3 (OK)
pandas >=1.1.1                :  1.4.1 (OK)
scipy >=0.17.0                :  1.8.0 (OK)
sympy >=0.7.3                 :  1.10.1 (OK)

[ba-tno]

This issue is resolved in pandas 1.4.3 and/or Spyder 5.3.2.

[ccordoba12]

I don’t know why this is happening, but since @dalthviz was able to reproduce it in terminal IPython, I think this is not a Spyder problem but a Pandas one. So the discussion should be continued on https://github.com/pandas-dev/pandas/issues/46890.

@ba-tno, @eddaz7, what do you say?