Suggestion:The encode conversion

Sir,I think over this question in django

UnicodeDecodeError at /admin/core/book/import/
'utf8' codec can't decode byte 0xc4 in position 0: invalid continuation byte

then I use vim set fileencoding,I found the csv standard of file exported from the django-import-export is UTF-8,I search it on internet,I also found that if i use utf-8 standard file to import,it’s OK.

Maybe I use the OS in Chinese language so it cause the problem.Would you please modify the project:If the encode of import-file or export-file is not utf-8,first convert it to utf-8,then process other code?

I have write some code to fix the encode convertion:

import chardet

def convertEncoding(from_encode,to_encode,old_filepath,target_file):
    f1=file(old_filepath)
    content2=[]
    while True:
        line=f1.readline()
        content2.append(line.decode(from_encode).encode(to_encode))
        if len(line) ==0:
            break

    f1.close()
    f2=file(target_file,'w')
    f2.writelines(content2)
    f2.close()

convertFile = open('1234.csv','r')
data = convertFile.read()
convertFile.close()

convertEncoding(chardet.detect(data)['encoding'], "utf-8", "1234.csv", "1234_bak.csv")

I am a newbie , my code is not concise.would you please think about that and integrite the regular to the project?I very like this project,thanks for your reputation!

[GabrieleCalarota]

Me too

Imported file has a wrong encoding: 'ascii' codec can't decode byte 0xc3 in position 31: ordinal not in range(128)

Can’t get it parsed with utf-8 in deploy environment, although in development works fine

[xiubinzheng]

Hi guys, Any updates on this, currently experiencing similar issues.