Headers parsed with added whitespace at the beginning
See original GitHub issue- Operating System: Windows 10 x64
- Node Version: 11.0.0
- NPM Version: 6.4.1
- csv-parser Version: 2.10
Expected Behavior
Example
let pokemonDescData = [];
fs.createReadStream('./data/csv/pokemon_desc.csv')
.pipe(csv(
{
separator: '`',
mapHeaders: (object) => console.log(object),
}))
.on('headers', (headers) => {
console.log(headers);
})
.on('data', (data) => pokemonDescData.push(
{
pokemon_id: data.pokemon_id,
p_desc: data.p_desc,
}
))
CSV file example
pokemon_id`p_desc
1`Bulbasaur can be seen nappin
2`There is a bud on this
Expected headers should be:
{ header: ‘pokemon_id’, index: 0 } { header: ‘p_desc’, index: 1 }
Actual Behavior
The data that is returned from the parsing will not contain ‘pokemon_id’, but contain ’ pokemon_id’.
If I use the mapHeaders function to trim() the headers then it works correctly.
Also, if I add an empty column with any dummy data, then the first column will have the whitespace, but the 2nd column, in this example ‘pokemon_id’ will be parsed correctly.
How Do We Reproduce?
Create a CSV file like:
pokemon_id`p_desc
1`Bulbasaur can be seen napping
2`There is a bud on this
Parse with csv({ separator: ‘`’ })
Issue Analytics
- State:
- Created 5 years ago
- Comments:7
Top Results From Across the Web
Handling whitespaces in http headers - Stack Overflow
Leading whitespace is not a problem, but trailing whitespace breaks streaming design. You can't parse HTTP 1.1 header in a pure streaming ...
Read more >The BIG-IP system may fail to properly parse HTTP headers ...
Security Advisory Description. The BIG-IP system may fail to properly parse HTTP headers that are prepended by whitespace. This issue occurs ...
Read more >How whitespace is handled by HTML, CSS, and in the DOM
There will be some text nodes that contain only whitespace, and; Some text nodes will have whitespace at the beginning or end.
Read more >Dealing with extra white spaces while reading CSV in Pandas
I'm creating a list containing a million rows and 3 columns: address with extra two blanks spaces at the beginning; name with start...
Read more >parse operator - Azure Data Explorer | Microsoft Learn
The * can't be used after a string type column. The parse pattern may start with ColumnName and not only with StringConstant. If...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
This may be useful to you (and others) facing similar unknowns: https://github.com/sindresorhus/strip-bom-stream. Using that, a stream can be stripped of the BOM before piping to csv-parser.
@shellscape Figured out the problem, it was a data corruption error.
I could not see the character in Notepad++ or VSCode, but looking at the .csv file in a hex editor, I found a few scrambled characters. Weird.
Sorry!