question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Access to parsed Tokens data on mapping error

See original GitHub issue

Thanks for the work done on this library! I’ve been looking at a few options to parse big csv files (2Gb+) and the performance of your implementation is remarkable.

I’m facing an issue now that could be a potential feature enhancement. At the end of the 2Gb files that I am parsing there are “footer” rows that can’t be Mapped to TEntity but they contain meaningful information on the file. Parsing the big file once being already time consuming, I’d like to avoid an other full parse. These lines are being mentioned in the Error collection but their Value is the same for all the footer lines [Column 2 is Out Of Range] and I can’t access the Raw data from there.

One possible fix would be to add a property with the raw data UnmappedRow to all error lines. I can think of a few other scenarios where a look through the raw data would be handy when an error occurs.

    public class CsvMappingError
    {
        public int ColumnIndex { get; set; }
        public string Value { get; set; }
        public string UnmappedRow { get; set; }
        public override string ToString()
        {
            return $"CsvMappingError (ColumnIndex = {ColumnIndex}, Value = {Value}, UnmappedRow = {UnmappedRow})";
        }
    }

And the CsvMapping.cs could add the raw data to the the Map() property. Something like:

Error = new CsvMappingError()
{
     ColumnIndex = columnIndex,
     Value = $"Column {columnIndex} is Out Of Range",
     UnmappedRow = string.Join("|", values.Tokens)
}

Alternatively, an UnmappedTokens porperty would also be helpful.

public IEnumerable<string> UnmappedTokens { get; set; }

Thank you

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:1
  • Comments:9 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
Bertrand-Sgcommented, Mar 21, 2019

Thanks for accepting the change and your quick replies!

1reaction
bytefishcommented, Mar 21, 2019

I have merged the Pull Request. Great, that you also wrote a Unit Test! It may take some time until the evening to build the new NuGet package (it’s 6 am here). Good job. 🤜🤛

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issue with parsing the content from JSON file with Jackson ...
Issue with parsing the content from JSON file with Jackson & message- JsonMappingException -Cannot deserialize as out of START_ARRAY token. ...
Read more >
Unable to Open Map "Invalid call with current token type"
You are getting this error because the Runtime, which both Explorer and Collector are built with, is having trouble opening/parsing the webmap.
Read more >
What Is JSON and How to Handle an “Unexpected Token” ...
Learn what JSON is and how you can deal with errors occurring when parsing JSON data, such as "Unexpected Token < in JSON...
Read more >
Failed to parse field of type in document with id ''.
A detailed guide on how to resolve errors related to "failed to parse field of type in document with id ''."
Read more >
OAuth2 access token response parsing fails with nested ...
An error occurred reading the OAuth 2.0 Access Token Response: JSON parse error: Cannot deserialize instance of `java.lang.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found