Make it easier to insert geometries, with documentation and maybe code

In playing with the new SpatiaLite helpers from #385 I noticed that actually populating geometry columns is still a little bit tricky. Here’s what I ended up doing:

import httpx, sqlite_utils
db = sqlite_utils.Database("/tmp/spatial.db")
attractions = httpx.get("https://latest.datasette.io/fixtures/roadside_attractions.json?_shape=array").json()
db["attractions"].insert_all(attractions, pk="pk")

# Schema of that table is now:
# CREATE TABLE [attractions] (
#    [pk] INTEGER PRIMARY KEY,
#    [name] TEXT,
#    [address] TEXT,
#    [latitude] FLOAT,
#    [longitude] FLOAT
# )

db.init_spatialite()
db["attractions"].add_geometry_column("point", "POINT")

db.execute("""
    update attractions set point = GeomFromText(
      'POINT(' || longitude || ' ' || latitude || ')', 4326
    )
""")

That last line took some figuring out - especially the need for the SRID of 4326, without which I got this error:

IntegrityError: attractions.point violates Geometry constraint [geom-type or SRID not allowed]

It would be good to both document this in more detail, but ideally also to come up with a more obvious pattern for inserting common types of spatial data.

Also related:

• #398
• #79

[simonw]

I wonder if I could implement the above such that this also works:

db["places"].insert(
    {
        "name": "London",
        "point": LongitudeLatitude(-0.118092, 51.509865)
    }
)

This feels like a very natural way to work with single inserts.

The challenge is writing the code inside .insert_all() such that it can handle these special objects in the input column values in addition to them being passed in conversions=.

I’m feeling very good about this direction in general though, it feels like it takes the existing but not particularly elegant conversions= mechanism and upgrades it to be far more useful, while maintaining backwards compatibility.

[eyeseast]

I like the idea of having stock conversions you could import. I’d actually move them to a dedicated module (call it sqlite_utils.conversions or something), because it’s different from other utilities. Maybe they even take configuration, or they’re composable.

from sqlite_utils.conversions import LongitudeLatitude

db["places"].insert(
    {
        "name": "London",
        "lng": -0.118092,
        "lat": 51.509865,
    },
    conversions={"point": LongitudeLatitude("lng", "lat")},
)

I would definitely use that for every CSV I get with lat/lng columns where I actually need GeoJSON.