encoding error reading back dataframe i just saved
See original GitHub issueWhen I try to read back a file saved using geopandas I get an encoding error. It works if I specify encoding=utf8 on save. However it would make more sense to fail on save. Otherwise you save some data and lose access because you cannot read it back.
gdf.to_file("popshapes.geojson", driver="GeoJSON")
gpd.read_file("popshapes.geojson")
Issue Analytics
- State:
- Created 4 years ago
- Comments:9 (6 by maintainers)
Top Results From Across the Web
UnicodeDecodeError when reading CSV file in Pandas with ...
You know the encoding, and there is no encoding error in the file. Great: you have just to specify the encoding: file_encoding =...
Read more >pandas.DataFrame.to_csv — pandas 1.5.2 documentation
Specifies how encoding and decoding errors are to be handled. See the errors argument for open() for a full list of options. New...
Read more >Error - unable to read the csv file in pandas
i am unable to read the csv file. import pandas as pd df=pd.read_csv(“data.csv”). the error i am getting is :- ...
Read more >Pandas read_csv() Tutorial: Importing Data - DataCamp
Importing Data is the first important step in any data science project. Learn how pandas' read_csv() function is perfect for this.
Read more >11 Data import - R for Data Science - Hadley Wickham
Both functions increase the chances of the output file being read back in correctly by: Always encoding strings in UTF-8. Saving dates and...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@simonm3 @martinfleis I’ve found the problem and am looking for it to be solved in GDAL/OGR: https://github.com/OSGeo/gdal/issues/2151. The way this affects fiona and GeoPandas is that this line https://github.com/Toblerity/Fiona/blob/maint-1.8/fiona/ogrext.pyx#L566 evaluates to False and so we fall back to the default encoding on your computer, which is presumably latin-1 or windows-1252.
It might be determined that fiona is misusing the OGR API, in which case we’ll need to make a special case for GeoJSON in that function.
Meanwhile, the workaround is to make sure that your environment’s default encoding is UTF-8 when you run your program.
This was most likely fixed in https://github.com/OSGeo/gdal/pull/2152. We can reopen if not.