[Proposal] Custom CSV Dialect

Use-case

• Data is exported to be imported into another application.
• Some applications only support one specific CSV dialect. (For example, the German tax consultancy software DATEV)
• We need to give developers and administrators the choice to provide the dialect.

This will support the DRY principle, removing the need for case-specific implementations like this one: https://github.com/frappe/erpnext/blob/master/erpnext/regional/report/fichier_des_ecritures_comptables_[fec]/fichier_des_ecritures_comptables_[fec].js#L91-L97

Proposal

1. Add Doctype “CSV Dialect” to frappe core.
2. Extend UnicodeWriter to consider CSV Dialect

1. Add Doctype “CSV Dialect”

Proposed location: frappe.core.doctype.csv_dialect

Screenshots:

Fields:

Name	Type	Default
Quoting	Select	Minimal
Quote Character	Data	"
Delimiter	Data	,
Doublequote	Checkbox	1
Escape Character	Data	None
Line Terminator	Data	\r\n
Write Header	Checkbox	1
Encoding	Data	utf-8

DocType:

https://github.com/alyf-de/frappe/blob/csv_dialect/frappe/core/doctype/csv_dialect/csv_dialect.json

2. Extend UnicodeWriter

Current: https://github.com/frappe/frappe/blob/bd2e410a02758733fd499273f497c13de25dced8/frappe/utils/csvutils.py#L96-L116

https://github.com/frappe/frappe/blob/d0e7690ae2f219c1688411a19f952179f885528d/frappe/public/js/frappe/misc/tools.js#L67-L77

Proposed: https://github.com/alyf-de/frappe/blob/csv_dialect/frappe/core/doctype/csv_dialect/csv_dialect.py

3. Give users the choice

In Report > Menu > Export > CSV, provide a link field to choose the CSV Dialect. Provided CSV shall conform to the dialect.

Result

Before a call to to_csv is made, we can ask the user to select a dialect which suits the target application. Required templates should come preinstalled and be set as default.

Thanks for reviewing. Maybe from @anandpdoshi who built the csv export?

[vrms]

I don’t think @anandpdoshi is still an (active) member here any longer. So we may need to find someone else to review this.

[rmehta]

Looks good 👍 maybe we can encapsulate the behaviour inside the CSV Dialect DocType