BUG: read_csv(index_col=False) -> auto-generated column headers for variable row length get headers of type int
See original GitHub issuePandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
from io import StringIO
csv = "1,2\n1,2,3"
io = StringIO(csv)
usecols = lambda header: print(type(header))
pd.read_csv(io, index_col=False, usecols=usecols)
# (..)
# <class 'int'>
Issue Description
When using index_col=False
kwarg for read_csv
then the number of columns read in is equal to the length of the longest row.
In such scenario, the columns which miss headers get names based on their position, but are assigned integers instead of strings. I think headers should rather be strings in general, but specifically this became a problem when I also used:
usecols=lambda header: header.strip() in headers
and it raised AttributeError.
Expected Behavior
The auto-generated column name/header is a string.
Installed Versions
INSTALLED VERSIONS
commit : 4bfe3d07b4858144c219b9346329027024102ab6 python : 3.8.10.final.0 python-bits : 64 OS : Linux OS-release : 5.4.0-89-generic Version : #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : pl_PL.UTF-8 LOCALE : pl_PL.UTF-8
pandas : 1.4.2 numpy : 1.22.3 pytz : 2022.1 dateutil : 2.8.2 pip : 20.0.2 setuptools : 45.2.0 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : None IPython : 8.3.0 pandas_datareader: None bs4 : 4.8.2 bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None markupsafe : 1.1.0 matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pyreadstat : None pyxlsb : None s3fs : None scipy : None snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None xlwt : None zstandard : None
Issue Analytics
- State:
- Created a year ago
- Comments:6 (4 by maintainers)
Top GitHub Comments
Ah got you, thanks.
You are correct, these should be strings, since we interpret all columns as strings anyway