CSV validation: is it RFC 4180 compliant?

Hi, if I run

frictionless validate "https://gist.githubusercontent.com/aborruso/3b675b529da264be83c7e75e26d2ee26/raw/e18c3cbbb5217acc430c950c3f5ae622eb52e661/tmp.csv"

I have “valid” result.

But in RFC 4180 I have “Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes.”

In the input I have the cell LISTA CIVICA | "SAINT-OYEN - TRAVAILLI EUNSEMBLO" and this is not a valid RFC 4180 cell. It should be "LISTA CIVICA | ""SAINT-OYEN - TRAVAILLI EUNSEMBLO""".

Then the question is: should frictionless validation be RFC 4180 compliant? If yes, this seems to me a bug.

Thank you

[aborruso]

Ok, I close it. Consider for yourself whether to include this validation exception in the documentation.

And thank you very much

[aivuk]

I agree with what @shashigharti said above, better to wait to see if a strict mode is implemented in the standard csv library or look for another library.

There is the library https://github.com/alan-turing-institute/CleverCSV that is a drop in substitute to the standard csv library and it can convert a csv to a RFC 4180 compliant one, but it also don’t raise an exception when a row is not compliant to the RFC. The library says that it can detect the dialect better with messy csv files:

With our method we achieve 97% accuracy for dialect detection, with a 21% improvement on non-standard (messy) CSV files compared to the Python standard library.

The only way that I can see to detect that the csv is no compliant to the RFC 4180 would be to change the standard csv library or the clevercsv (in that case it would need to be done in the C parser https://github.com/alan-turing-institute/CleverCSV/blob/master/src/cparser.c). Other idea would be to just convert the csv to a standard compliant one using clevercsv and comparing the different csv files, but this would not be recommended for big csv files.