question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Headers parsed with added whitespace at the beginning

See original GitHub issue
  • Operating System: Windows 10 x64
  • Node Version: 11.0.0
  • NPM Version: 6.4.1
  • csv-parser Version: 2.10

Expected Behavior

Example

    let pokemonDescData = [];
    fs.createReadStream('./data/csv/pokemon_desc.csv')
        .pipe(csv(
            {
              separator: '`',
              mapHeaders: (object) => console.log(object),
            }))
        .on('headers', (headers) => {
          console.log(headers);
        })
        .on('data', (data) => pokemonDescData.push(
            {
              pokemon_id: data.pokemon_id,
              p_desc: data.p_desc,
            }
        ))

CSV file example

pokemon_id`p_desc
1`Bulbasaur can be seen nappin
2`There is a bud on this

Expected headers should be:

{ header: ‘pokemon_id’, index: 0 } { header: ‘p_desc’, index: 1 }

Actual Behavior

image

The data that is returned from the parsing will not contain ‘pokemon_id’, but contain ’ pokemon_id’.

If I use the mapHeaders function to trim() the headers then it works correctly.

Also, if I add an empty column with any dummy data, then the first column will have the whitespace, but the 2nd column, in this example ‘pokemon_id’ will be parsed correctly.

How Do We Reproduce?

Create a CSV file like:

pokemon_id`p_desc
1`Bulbasaur can be seen napping
2`There is a bud on this

Parse with csv({ separator: ‘`’ })

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:7

github_iconTop GitHub Comments

1reaction
shellscapecommented, Nov 5, 2018

This may be useful to you (and others) facing similar unknowns: https://github.com/sindresorhus/strip-bom-stream. Using that, a stream can be stripped of the BOM before piping to csv-parser.

1reaction
DryWaterscommented, Nov 2, 2018

@shellscape Figured out the problem, it was a data corruption error.
I could not see the character in Notepad++ or VSCode, but looking at the .csv file in a hex editor, I found a few scrambled characters. Weird.
Sorry!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling whitespaces in http headers - Stack Overflow
Leading whitespace is not a problem, but trailing whitespace breaks streaming design. You can't parse HTTP 1.1 header in a pure streaming ...
Read more >
The BIG-IP system may fail to properly parse HTTP headers ...
Security Advisory Description. The BIG-IP system may fail to properly parse HTTP headers that are prepended by whitespace. This issue occurs ...
Read more >
How whitespace is handled by HTML, CSS, and in the DOM
There will be some text nodes that contain only whitespace, and; Some text nodes will have whitespace at the beginning or end.
Read more >
Dealing with extra white spaces while reading CSV in Pandas
I'm creating a list containing a million rows and 3 columns: address with extra two blanks spaces at the beginning; name with start...
Read more >
parse operator - Azure Data Explorer | Microsoft Learn
The * can't be used after a string type column. The parse pattern may start with ColumnName and not only with StringConstant. If...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found