question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Warning messages to add in the CSV exporter when there's any field dropped

See original GitHub issue

Summary

A warning message shall be generated when there’re some fields dropped within CsvItemExporter (or also in other exporters?) during exporting. We may have this kind of warning messages emitted only for the first occurrence to avoid generating too much noise, similar to the warning messages when there’s possible data loss (related code).

Motivation

Due to nature of the CSV format, the CsvItemExporter needs to determine the list of fields to write before writing the 1st record (related code), which would in some occasions cause unintentional loss of data.

Unfortunately I’ve yet observed this kind of mistakenly introduced data loss issue from different developers, where the common cause may be:

  • Returning a dict as the 1st item that doesn’t have all fields included.
  • Returning more than one type of items that have different fields.

(both when not having FEED_EXPORT_FIELDS properly configured)

Describe alternatives you’ve considered

Properly configuring the FEED_EXPORT_FIELDS setting entry would surely have the issue resolved. However that’s not always guaranteed (a developer may just forget to, etc.).

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:1
  • Comments:12 (11 by maintainers)

github_iconTop GitHub Comments

1reaction
Gallaeciocommented, Sep 30, 2019

But we only want such a warning logged if there is actual data loss, I think.

0reactions
jtboingcommented, Jan 17, 2021

Has this issue been resolved? If so, then maybe this issue can be closed.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Common problems when importing or exporting respondents
One or more of your respondent list fields contains too many characters and/or contains invalid characters. · There is an error in the...
Read more >
CSV export problem, multiple points exported, some not in ...
If I shut down TBC and reopen it will export correctly, but it is very disconcerting and could cause serious problems in the...
Read more >
SFTP Syncs: Troubleshooting CSV formatting errors
Check that the file is in CSV format. The following file includes missing commas: [file] [lines], Ensure there are no missing commas in...
Read more >
CSV Export of all fields failing with network error | Jira
The CSV export was failing when selecting all fields because one of the Custom Fields was corrupted. On this specific sample, the custom...
Read more >
Flat file potential data loss issue using SQL server import wizard
I have a csv with 6 columns ,one of the field name is "impactdate" ... There may be error messages posted before this...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found