question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

STATUS_INVALID_PARAMETER: 0xc000000d when reading CSV with pandas

See original GitHub issue

I have an issue where, while reading a CSV file with smbprotocol using pandas.read_csv, I receive the following Exception:

Traceback (most recent call last):
  File "/home/matt/code/smb-example-python/smb.py", line 50, in <module>
    df = pd.read_csv(fd)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/pandas/io/parsers.py", line 610, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/pandas/io/parsers.py", line 468, in _read
    return parser.read(nrows)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/pandas/io/parsers.py", line 1057, in read
    index, columns, col_dict = self._engine.read(nrows)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/pandas/io/parsers.py", line 2061, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 756, in pandas._libs.parsers.TextReader.read
  File "pandas/_libs/parsers.pyx", line 771, in pandas._libs.parsers.TextReader._read_low_memory
  File "pandas/_libs/parsers.pyx", line 827, in pandas._libs.parsers.TextReader._read_rows
  File "pandas/_libs/parsers.pyx", line 814, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 1943, in pandas._libs.parsers.raise_parser_error
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/smbclient/_io.py", line 573, in readinto
    file_bytes = recv_func(request)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/smbprotocol/open.py", line 1313, in _read_response
    response = self.connection.receive(request, wait=wait)
  File "/home/matt/code/smb-example-python/env/lib/python3.8/site-packages/smbprotocol/connection.py", line 928, in receive
    raise SMBResponseException(response)
smbprotocol.exceptions.InvalidParameter: Received unexpected status from the server: An invalid parameter was passed to a service or function. (3221225485) STATUS_INVALID_PARAMETER: 0xc000000d

This is generated from the following code (sanitised to remove private info):

from smbclient import (
    open_file,
    register_session,
)
import pandas as pd

register_session("server", username=username, password=password)

with open_file(r"//server/SHARE/path/to/file.csv") as fd:
    df = pd.read_csv(fd)
    print(df)

From debugging, I can see that there are several successful calls to file_bytes = recv_func(request) in readinto(), where contents of the file are read successfully, and part way through the file read, the exception above occurs.

Some other things I’ve observed:

  • Several other similar CSV files in the same file share can be read successfully. So there appears to be something particular about this file, but I cannot see anything particular at the offset where the error occurs. (I’m afraid I can’t share the file).
  • If I read the file from a local directory using pd.read_csv(), I can read the file successfully.
  • If I switch the pandas parser using pd.read_csv(engine="python") then I can read the file successfully using smbprotocol`, so there seems to be something about the interaction between the pandas C parser and smbprotocol.
  • If I skip pandas and just do print(fd.read()) I can read the whole file, which also suggests something about the interaction between the pandas C parser and smbprotocol.
  • At the time of the failure, the chunksize is 262145 and the credit_request is 5. This is the first time the credit_request is greater than 4 in the iterations that precede the failure.

I realise I’m probably not providing enough information to get a definitive cause. But any advice on how to proceed in debugging this issue would be gratefully received.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
mattinbitscommented, Jun 22, 2021

After doing some debugging on the read calls that pandas is making, I can reproduce the error without pandas:

with open_file("//localhost/public/records.csv") as fd:
    while True:
        fd.read(262144)
0reactions
mattinbitscommented, Aug 5, 2021

Thank you, it’s working perfectly.

Read more comments on GitHub >

github_iconTop Results From Across the Web

0xc000000d when reading CSV with pandas - Bountysource
I have an issue where, while reading a CSV file with smbprotocol using pandas.read_csv , I receive the following Exception:
Read more >
Error while loading data from csv to table using pandas ...
From the error that you got: pyodbc.ProgrammingError: ('Invalid parameter type. param-index=0 param-type=numpy.int64', 'HY105').
Read more >
Python | Read csv using pandas.read_csv() - GeeksforGeeks
Parameters : filepath_or_buffer: It is the location of the file which is to be retrieved using this function. It accepts any string path...
Read more >
Pandas read_csv() - How to read a csv file in Python
Explains different ways pandas read_csv function can be used to read csv files into pandas dataframe, along with examples.
Read more >
How To Fix pandas.parser.CParserError: Error tokenizing data
The error may also be related to the delimiters and/or headers (not) specified when calling read_csv . Make sure to pass both the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found