Incorrect detection of encoding gives out a source-error

We have a csv file that uses utf-8 encoding, but uses only ascii characters up to some point at the middle of the file. Goodtables is probably incorrectly interpreting it as an ascii file, because it gives out a source-error:

$ goodtables --infer-schema data/nz.csv 
DATASET
=======
{'error-count': 1,
 'preset': 'nested',
 'table-count': 1,
 'time': 0.003,
 'valid': False}

TABLE [1]
=========
{'error-count': 1,
 'headers': [],
 'row-count': 0,
 'source': 'data/nz.csv',
 'time': 0.002,
 'valid': False}
---------
[-,-] [source-error] 'charmap' codec can't decode byte 0x81 in position 1780: character maps to <undefined>

File: data/nz.csv

The line that gives an error is L18:

nz/archives-new-zealand-te-rua-mahara-o-te-kwanatanga,Archives New Zealand Te Rua Mahara o te Kāwanatanga,,,,,,,,,http://www.archives.govt.nz,NZ,,,,,

Apparently, this would have been fixed in #45 by implementing a decode_strategy keyword. But I could not find any mention of it in the documentation. The default settings still leave users of Goodtables confused.

[AntoineAugusti]

I can confirm

goodtables.validate(
    [{"source": "exemple-valide.csv", "schema": "schema.json", "encoding": "utf-8"}]
)

works as expected. Thanks!

[roll]

It works only if encoding=utf-8 is provided. We’re going to improve quality of detection here - https://github.com/frictionlessdata/tabulator-py/issues/308.

For now, as it’s a tabulator/chardet problem source-error is a proper error code from goodtables.