question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

read_fwf: skip_blank_lines does nothing

See original GitHub issue

Code Sample, a copy-pastable example if possible

from io import StringIO
import pandas as pd

f = StringIO('''A B

C D''')

df = pd.read_fwf(f, colspecs=[(0, 1), (2,3)], header=None, skip_blank_lines=True)
print(df)

Problem description

Output:

     0    1
0    A    B
1  NaN  NaN
2    C    D

The (second) blank line is not skipped, but instead there is a row with two NaN values. It seems that skip_blank_lines has no effect on read_fwf. On the other hand, read_csv(f, sep=' ', header=None), produces the expected output below.

Expected Output

   0  1
0  A  B
1  C  D

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None python: 3.6.6.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 79 Stepping 1, GenuineIntel byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None

pandas: 0.23.1 pytest: 3.6.2 pip: 18.0 setuptools: 39.0.1 Cython: None numpy: 1.15.1 scipy: None pyarrow: None xarray: None IPython: 6.5.0 sphinx: 1.5.5 patsy: None dateutil: 2.7.3 pytz: 2018.5 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: 1.0.1 sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.10 s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
jrebackcommented, Jun 4, 2020

@dcdenu4 or anyone can submit a PR pandas is all volunteer and we have 3000+ open issues

0reactions
jncltcommented, Dec 2, 2022

As mentioned above, the original issue isn’t reproducible anymore and there is a test covering this case (test_fwf_skip_blank_lines): https://github.com/pandas-dev/pandas/blob/36dcf519c67a8098572447f7d5a896740fc9c464/pandas/tests/io/parser/test_read_fwf.py#L354

As for read_csv, I think the behavior is as expected (Nan for coma-separated blank values).

I guess this issue can be closed?

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to skip blank lines with read_fwf in pandas?
It looks like the skip_blank_lines=True argument is ignored, as the dataframe contains NaN's. What should be the valid combination of pandas.
Read more >
[Solved]-Why is pandas.read_fwf not skipping the blank line as ...
One missing detail in your code is that you failed to pass widths parameter. But this is not all. Another problem is that...
Read more >
pandas.read_fwf — pandas 0.20.3 documentation
Intervening rows that are not specified will be skipped (e.g. 2 in this example is skipped). Note that this parameter ignores commented lines...
Read more >
Read a fixed width file into a tibble — read_fwf • readr
read_fwf ( file, col_positions = fwf_empty(file, skip, n = guess_max), ... If NULL , all column types will be imputed from guess_max rows...
Read more >
How to skip blank lines with read_fwf in pandas? - YouTube
Pandas : How to skip blank lines with read_fwf in pandas? [ Beautify Your Computer : https://www.hows.tech/p/recommended.html ] Pandas : How ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found