Issue with CSV ingestion with non-default delimiter

The code assumes that the delimiter for CSV in the reader config should also be used when specifying the header string in the ingestion spec. While this is (arguably) a valid assumption, it might be non intuitive for the user since these are two different configs in two different places.

Filing this issue for a better user experience, perhaps:

• If the CSV delimiter does not work for header String in job spec, try comma?
• At the very least, error out notifying the user that the two are inconsistent.

[suddendust]

@mayankshriv I think gracefully erroring out if they don’t match is the least we can do here. Shall we start with that?

[mayankshriv]

@suddendust: In the ingestion job spec yaml file, we provide delimiter, header as two configs. It is implicitly assumed that header strings will also be separated by delimiter.