question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TST: read_csv silently drops empty row

See original GitHub issue

While going over some tests, I noticed the following here:

        df = DataFrame([[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12]],
                       index=['one', 'two'],
                       columns=MultiIndex.from_tuples(
                           [('a', 'q'), ('a', 'r'), ('a', 's'),
                            ('b', 't'), ('c', 'u'), ('c', 'v')]))


        # to_csv
        data = """,a,a,a,b,c,c
,q,r,s,t,u,v
,,,,,,
one,1,2,3,4,5,6
two,7,8,9,10,11,12"""


        result = self.read_csv(StringIO(data), header=[0, 1], index_col=0)
        tm.assert_frame_equal(df, result)

Is this really expected behaviour, that pandas.read_csv should silently discard an empty row following the header if multiple rows are passed as headers?

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:1
  • Comments:12 (11 by maintainers)

github_iconTop GitHub Comments

3reactions
WillAydcommented, Jul 20, 2018

Well there is a parameter for skip_blank_lines which is True by default, but setting it to False didn’t change your example which seems off.

Investigation and PRs certainly welcome!

0reactions
gfyoungcommented, Nov 7, 2022

@Michael-E-Rose: since the parameters are relatively close, I would keep here for now.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pandas read_csv: ignore trailing lines with empty data
Some clarifications: The lines of empty data will always been trailing, it's a common problem with csv files generated from Excel.
Read more >
pandas.read_csv: how to skip empty lines
Let us suppose that we start with a CSV file that has empty rows: ... import pandas as pd >>> df = pd.read_csv("test.csv", ......
Read more >
Changelog - readr - Tidyverse
csv and friends are new example datasets accessible via readr_example() , which have been added to illustrate reading multiple files at once, into...
Read more >
pandas check if cell is empty - Coffeenote
... read_csv. Default: empty; silent (bool, optional) – Suppress all stderr output. ... Example 1: An empty dataframe with 0 rows and 0...
Read more >
delete rows in csv file python - You.com | The AI Search ...
I found the problem. row[0] in your code returns the entire row, that means the lines are not parsed correctly. After a bit...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found