spurious SQL UPDATE being emitted when Geography has not changed

@zzzeek fresh from our recent brief but efficient collaboration on https://github.com/sqlalchemy/alembic/pull/636 i thought i’d file this issue here but with this disclaimer: I’m conscious that the root cause of this issue might lie within geoalchemy2. apologies in advance if this turns out to be the case.

observed behaviour: i’m using sqlalchemy 1.3.11 with a PostgreSQL/PostGIS backend i have a table with a Geography column that represents a GIS point i retrieve row from that table i update the GIS point with the same lat/lon values as before i observe an unexpected SQL update is issued to the database

exposition in code:

from datetime import datetime
from decimal import Decimal

import geoalchemy2
from geoalchemy2 import Geography
from shapely.geometry import Point
from sqlalchemy.event import listens_for
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import (
    class_mapper,
    sessionmaker)
from sqlalchemy.orm.session import Session
from sqlalchemy import (
    create_engine,
    inspect,
    Column,
    Integer,
    String, Numeric, DateTime)


engine = create_engine('<database-connection-string>', echo=True)
session = sessionmaker(bind=engine, autoflush=False)()
Base = declarative_base()


class Place(Base):
    __tablename__ = "place"
    id = Column('ID', Integer, primary_key=True)
    name = Column('NAME', String(50))
    latitude = Column('LATITUDE', Numeric(10, 8), nullable=False)
    longitude = Column('LONGITUDE', Numeric(11, 8), nullable=False)
    geo_point = Column('GEO_POINT', Geography(geometry_type='POINT', srid=4326), nullable=False)
    modified = Column('MODIFIED', DateTime(), nullable=False)


@listens_for(Place, "before_update")
def before_update(mapper, connection, target):
    session = Session.object_session(target)

    if session.is_modified(target, include_collections=False):
        print('target is modified')
        target.modified = datetime.utcnow()
        inspection_object = inspect(target)
        attrs = class_mapper(target.__class__).column_attrs
        for attr in attrs:
            hist = getattr(inspection_object.attrs, attr.key).history
            print("attribute {} has changes: {}".format(attr.key, hist.has_changes()))

            if len(hist.added):
                print("added: {}".format(hist.added[0]))

            if len(hist.deleted):
                print("deleted: {}".format(hist.deleted[0]))
    else:
        print('target is NOT modified')


with engine.begin() as connection:
    metadata = Base.metadata
    metadata.create_all(connection)

session.execute('TRUNCATE TABLE place')

print("\n\nAdding a place")
place = Place()
place.latitude = Decimal('{0:2.8f}'.format(52.238049))
place.longitude = Decimal('{0:3.8f}'.format(6.0916571111111))
place.geo_point = geoalchemy2.shape.from_shape(Point(place.longitude, place.latitude), srid=4326)
place.name = 'placename'
place.modified = datetime.utcnow()
session.add(place)
session.commit()
session.close()

print("\n\nBeginning new session")
session = sessionmaker(bind=engine, autoflush=False)()

print("\n\nFinding a place")
place = session.query(Place).filter(Place.name == 'placename').one()

print("\n\nUpdating a place")
place.latitude = Decimal('{0:2.8f}'.format(52.238049))
place.longitude = Decimal('{0:3.8f}'.format(6.0916571111111))
place.geo_point = geoalchemy2.shape.from_shape(Point(place.longitude, place.latitude), srid=4326)
place.name = 'placename'
session.add(place)
session.flush()

and the [partial] output from this:

2020-01-13 12:41:46,622 INFO sqlalchemy.engine.base.Engine TRUNCATE TABLE place
2020-01-13 12:41:46,622 INFO sqlalchemy.engine.base.Engine {}


Adding a place
2020-01-13 12:41:46,649 INFO sqlalchemy.engine.base.Engine INSERT INTO place ("NAME", "LATITUDE", "LONGITUDE", "GEO_POINT", "MODIFIED") VALUES (%(NAME)s, %(LATITUDE)s, %(LONGITUDE)s, ST_GeomFromWKB(%(ST_GeomFromWKB_1)s, %(ST_GeomFromWKB_2)s), %(MODIFIED)s) RETURNING place."ID"
2020-01-13 12:41:46,649 INFO sqlalchemy.engine.base.Engine {'NAME': 'placename', 'LATITUDE': Decimal('52.23804900'), 'LONGITUDE': Decimal('6.09165711'), 'ST_GeomFromWKB_1': <memory at 0x7fc7d06eb7c8>, 'ST_GeomFromWKB_2': 4326, 'MODIFIED': datetime.datetime(2020, 1, 13, 11, 41, 46, 648390)}
2020-01-13 12:41:46,651 INFO sqlalchemy.engine.base.Engine COMMIT


Beginning new session


Finding a place
2020-01-13 12:41:46,659 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
2020-01-13 12:41:46,661 INFO sqlalchemy.engine.base.Engine SELECT place."ID" AS "place_ID", place."NAME" AS "place_NAME", place."LATITUDE" AS "place_LATITUDE", place."LONGITUDE" AS "place_LONGITUDE", ST_AsBinary(place."GEO_POINT") AS "place_GEO_POINT", place."MODIFIED" AS "place_MODIFIED" 
FROM place 
WHERE place."NAME" = %(NAME_1)s
2020-01-13 12:41:46,661 INFO sqlalchemy.engine.base.Engine {'NAME_1': 'placename'}


Updating a place
target is modified
attribute id has changes: False
attribute name has changes: False
attribute latitude has changes: False
attribute longitude has changes: False
attribute geo_point has changes: True
added: 01010000006095875cdb5d184039ecbe63781e4a40
deleted: 01010000006095875cdb5d184039ecbe63781e4a40
attribute modified has changes: True
added: 2020-01-13 11:41:46.666385
deleted: 2020-01-13 11:41:46.648390
2020-01-13 12:41:46,667 INFO sqlalchemy.engine.base.Engine UPDATE place SET "GEO_POINT"=ST_GeomFromWKB(%(ST_GeomFromWKB_1)s, %(ST_GeomFromWKB_2)s), "MODIFIED"=%(MODIFIED)s WHERE place."ID" = %(place_ID)s
2020-01-13 12:41:46,668 INFO sqlalchemy.engine.base.Engine {'ST_GeomFromWKB_1': <memory at 0x7fc7d06eb888>, 'ST_GeomFromWKB_2': 4326, 'MODIFIED': datetime.datetime(2020, 1, 13, 11, 41, 46, 666385), 'place_ID': 16}

Process finished with exit code 0

I draw attention to the section

attribute geo_point has changes: True
added: 01010000006095875cdb5d184039ecbe63781e4a40
deleted: 01010000006095875cdb5d184039ecbe63781e4a40

I’ve done my best to understand the issue by stepping through the code. The key stage seems to be the call to from_scalar_attribute within class History in sqlalchemy.orm.attributes on line 1655. I confess to losing full grasp of the flow once the step-through goes inside attribute.is_equal(current, original). I would expect this to return True, but it doesn’t.

If this is a legitimate issue, I’d be happy to assist with producing a fix, if it’s within my abilities. I’d also be grateful for any suggested workarounds.

[matthew-emw]

many thanks for your guidance. implementing a __eq__() as you describe makes the behaviour conform to my expectations. i’m going to head on over to geoalchemy2, reference this discussion and see what can be done. thanks again.

[zzzeek]

this is over on the geoalchemy side. I dont think SQLAlchemy should add additional hooks to guess if an object that intends to be both SQL and a value at the same time is one or the other, IMO these two kinds of objects are separate.