Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to detect the encoding of the loaded file before displaying data ?

See original GitHub issue

Hi !

I had a problem for displaying special characters with CVSReader due to csv files which were encoded as ISO-8859-1 instead of utf-8 (ex : Pyr�n�es-Atlantiques )

By adding :

    encoding: "ISO-8859-1",

in my code it’s working, the browser can now read special characters without problems. But the new problem is when loading a file encoded in utf-8, the characters are not displayed properly (ex : Pyrénées-Atlantiques )

My problem is that I am working with clients who do not use the same encoding for their csv files. Some clients use “utf-8”, others use “ISO-8859-1”. And I can not know in advance what will be the encoding of the file used.

Here is my code :

let changeEncoding = false 
   const  handleOnFileLoad = (data) => { => {
      if ( => element.includes("�"))) {
        changeEncoding = true
    if (changeEncoding) {
      alert("Some characters of your file will not display properly. Please load again yout file.")
    } else {
    data = data.slice(1)
    const enrollFieldArray = {}
    enrollFieldArray["data"] = =>

I use redux and the “encoding” variable is in the state with this default value :

encoding: {encoding: "UTF-8"}

With this solution, the client must load the file a first time to update the state with the correct value of “encoding”, and load the file a second time to display data with the correct encoding.

Is there a native CSVReader’s method that allows you to detect the encoding of the loaded file before displaying the data ?

Thank you very much for your help


Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

exaucaecommented, Sep 7, 2021

I may send a pull request in the coming weeks!

exaucaecommented, Oct 14, 2021

@andirkh ,thanks for the bump. Pull request is #100. Feedbacks welcomed!

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to detect the encoding of a file?
There is a pretty simple way using Firefox. Open your file using Firefox, then View > Character Encoding. Detailed here.
Read more >
How To Find Out What Character Encoding Is Used In a ...
Profiling Character Encoding. Files in UTF-8, ISO 8859-1, ASCII, XML, GML. Impact on Data Quality. Free character profiling tool.
Read more >
python - How to determine the encoding of text - Stack Overflow
Correctly detecting the encoding all times is impossible. ... '''Predict a file's encoding using chardet''' # Open the file as binary data ......
Read more >
Encoding | JetBrains Rider Documentation
With the file open in the editor, either select File | File Properties | File Encoding from the main menu or click the...
Read more >
Processing Text Files in Python 3 - Nick Coghlan's Python Notes
surrogateescape : this is the error handler that Python uses for most OS facing APIs to gracefully cope with encoding problems in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found