question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

column names with leading underscores with df.itertuples cause AttributeError when accessed

See original GitHub issue

Code Sample, a copy-pastable example if possible

data = [['Alex',10],['Bob',12],['Clarke',13]]
df = pd.DataFrame(data, columns=['_name','age'])
try:
    for row in df.itertuples():
        val = getattr(row, '_name')
        print(val)
except AttributeError:
    print('_name column not accessible')

Problem description

DataFrame columns named with a leading underscore are not accessible. While it is possible to rename the column to access the data, Pandas users unfamiliar with this issue might spend a bit of time developing their own workaround. The perplexing thing about this is when the column names are output for inspection you can see the column name with the leading underscore, e.g. print(df.dtypes.index)

[this should explain why the current behaviour is a problem and why the expected output is a better solution.]

Note: We receive a lot of issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates!

Note: Many problems can be resolved by simply upgrading pandas to the latest version. Before submitting, please check if that solution works for you. If possible, you may want to check if master addresses this issue, but that is not necessary.

For documentation-related issues, you can check the latest versions of the docs on master here:

https://pandas-docs.github.io/pandas-docs-travis/

If the issue has not been resolved there, go ahead and file it in the issue tracker.

Expected Output

The expected output is the data from the column whose name begins with an underscore.

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line] INSTALLED VERSIONS

commit: None python: 3.6.5.final.0 python-bits: 64 OS: Windows OS-release: 2012ServerR2 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.23.4 pytest: None pip: 10.0.1 setuptools: 39.0.1 Cython: None numpy: 1.15.3 scipy: None pyarrow: None xarray: None IPython: None sphinx: None patsy: None dateutil: 2.7.5 pytz: 2018.7 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None sqlalchemy: 1.2.6 pymysql: None psycopg2: None jinja2: 2.10 s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
TomAugspurgercommented, Feb 2, 2019

@gaetano-guerriero that’s a bug. Doing a release today with a fix.

1reaction
gaetano-guerrierocommented, Feb 2, 2019

I think it would be important to document that the silent renaming can happen elsewhere:

>>> df = pd.DataFrame({'a': [1, 2], '_b': [3, 4]})
>>> df.to_dict(orient='records')
[{'a': 1, '_1': 3}, {'a': 2, '_1': 4}] 
Read more comments on GitHub >

github_iconTop Results From Across the Web

Accessing pandas cell value using df.itertuples() and column ...
1 Answer 1 · This throws the following error: TypeError: tuple indices must be integers or slices, not str My actual column names...
Read more >
How to iterate over rows in Pandas: Most efficient options
There are many ways to iterate over rows of a DataFrame or Series in pandas, each with their own pros and cons. Since...
Read more >
The Meaning of Underscores in Python - dbader.org
However, leading underscores do impact how names get imported from modules. ... A double underscore prefix causes the Python interpreter to rewrite the ......
Read more >
pandas.DataFrame.itertuples — pandas 1.5.2 documentation
The column names will be renamed to positional names if they are invalid Python identifiers, repeated, or start with an underscore. Examples. >>>...
Read more >
Tutorial: How to Index DataFrames in Pandas - Dataquest
Since both the dataframe index and column names contain only ... name contains a punctuation mark (except for the underscore) df.col-7
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found