question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Writing to GeoPackage doesn't check for SQL-protected words in column names or "forbidden" characters in layer names

See original GitHub issue

Expected behavior and actual behavior.

When creating a GPKG file, there are a few safe-checks that are missing. The two kinds of problems I’ve identified are:

  1. Column names that use SQL protected words, like select and count are currently allowed
  2. Layer names that contain hyphens are allowed

Currently, Fiona allows you to generate GeoPackage files that violate these rules. However, when you try to open files that violate these rules in ArcGIS Pro, the file fails to load and you just get a non-descriptive error message.

My expectation is that the columns should be automatically renamed to avoid clashes with SQL-protected words. Offending columns can have an underscore added to the end.

Furthermore, layer names should automatically transform hyphens into underscores.

Note: Since I mainly use Fiona through GeoPandas, I’ve also created an issue on their Github.

Note 2: Apologies for the example below using GeoPandas

Steps to reproduce the problem.

import geopandas as gpd

test_gdf = gpd.GeoDataFrame({'my_id':[1,2,3],
                             'select':[1,2,3],
                             'count':[1,2,3]},
                            geometry=gpd.points_from_xy([0,1,2], [0,1,2]),
                            crs='epsg:4326')

test_gdf.to_file('test.gpkg', driver='GPKG',layer='with-hyphen')

Then, try to open the with-hyphen layer from the test.gpkg' file in ArcGIS Pro. An error message will pop up.

Operating system

Windows 10

Fiona and GDAL version and provenance

fiona: 1.8.13.post1 (installed through Anaconda, I think) GDAL: 3.0.2 (installed through Anaconda

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
d-wassermancommented, Jul 5, 2021

Understood. That makes sense to me. It felt like a stretch as a suggestion.

1reaction
rbuffatcommented, Apr 22, 2021

select * from "test-table" seems also to work. Note that this is with PostGIS and not GPKG/SQLite.

I don’t want to rule out that there are restrictions, I’m just at the moment not aware of any.

Read more comments on GitHub >

github_iconTop Results From Across the Web

BUG: Writing to GeoPackage doesn't check for SQL-protected ...
BUG: Writing to GeoPackage doesn't check for SQL-protected words in column names or "forbidden" characters in layer names #1914.
Read more >
Changing layer names when converting from WFS to ...
I want to convert WFS to GeoPackage, everything ...
Read more >
How do I escape reserved words used as column names ...
String escaping in ANSI SQL is done by using double quotes ("). Unfortunately, this escaping method is not portable to MySQL, unless it...
Read more >
Field name restrictions in a geodatabase table - Esri Community
I wanted to check to see what the restrictions were for field names in a geodatabase table. Alphanumeric characters only, 64 characters in ......
Read more >
NEWS - OSGeo
Credit to OSS Fuzz ODBC driver: * deal with table names that require ... is present but invalid (#6976) * fix logic to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found