Back reference is not available after session end

Describe the bug In an eager loading setup, back references are not accessible outside of session block (while inside the block no additional SQL queries are being produced).

Expected behavior Outside of session block the back reference is still available.

To Reproduce https://gist.github.com/Randelung/81e0ccacc28c8ed1c205c3da799b3882 Have classes referencing each other, back_populates and backref in relationship both produce the same result. The example builds a tree structure of four types A, B, C, and D, and fills them with three children for each level. Load the structure using eager loading (joined is used in example, selectin produces same result). Try to access the back reference (done via print and __repr__) both inside and outside the session block. It prints fine inside the session block, even though it’s not fetching any more data from the database, but throws an error when outside (move print statement).

Error

Traceback (most recent call last):
  File "D:\Desktop\work\VL\test\script.py", line 61, in <module>
    asyncio.run(main())
  File "C:\Users\rando\AppData\Local\Programs\Python\Python39\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Users\rando\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete     
    return future.result()
  File "D:\Desktop\work\VL\test\script.py", line 58, in main
    print(result)
  File "D:\Desktop\work\VL\test\script.py", line 27, in __repr__
    return f'A({self.id},{self.name},{self.bs})'
  File "D:\Desktop\work\VL\test\script.py", line 38, in __repr__
    return f'B({self.id},{self.name},{self.a.id})'
  File "C:\Users\rando\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\orm\attributes.py", line 449, in __get__
    return self.impl.get(state, dict_)
  File "C:\Users\rando\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\orm\attributes.py", line 893, in get    value = self.callable_(state, passive)
  File "C:\Users\rando\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\orm\strategies.py", line 827, in _load_for_state
    raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <B at 0x2280d585d30> is not bound to a Session; lazy load operation of attribute 'a' cannot proceed (Background on this error at: http://sqlalche.me/e/14/bhk3)

Versions.

• OS: Win10
• Python: 3.9.2
• SQLAlchemy: 1.4
• Database: MariaDB 10.5
• DBAPI: 2.0

Have a nice day!

Edit: simpler test case means shorter error log. Same error, though.

[zzzeek]

I woudl say we can add some sub-bullets to the first bullet at https://docs.sqlalchemy.org/en/14/orm/extensions/asyncio.html#synopsis-orm indicating “other loader options that are helpful include: immediateload, joinedload, subqueryload”

[zzzeek]

great this is a simple issue. joinedload at the relationship() level will not automatically traverse loops, because it leads to a cycle between the two tables that is inherently wasteful - think of “SELECT A, B, A1 FROM A join B join A AS A1” - not a good look.

instead, those many to ones are in the identity map because they were just loaded as a collection. one way to make this work is to use “immediateload” and the “lazy load” will trigger immediately inline with the query, which is not actually a “load” at all because it pulls from the identity map:

class B(Base):
    __tablename__ = 'b'

    id = Column('id', Integer, autoincrement=True, primary_key=True)
    a_id = Column('a', Integer, ForeignKey('a.id'))
    a = relationship('A', back_populates='bs', lazy='immediate')
    name = Column('name', String(255))

    def __repr__(self) -> str:
        return f'B({self.id},{self.name},{self.a.id})'

using the above technique, you get only one SELECT, the first one for A + A->B. the many to ones don’t use a SELECT or a JOIN or anything. Depending on what you are doing and which side you are loading from, you may need to use loader options at query time to set this up, because if you load a lot of Bs for many different As you would see a SELECT statement per unique A emitted, unless you set up some options.

Given that “selectinload” is much better for collections, that leads to the next approach, use “selectinload” on the collection side then stick with “joinedload” on the many-to-one. When the “selectinload” takes place, the “joinedload” for your many-to-ones works too. This also means you can safely load from the many-to-one side only and have eager loading in both directions. The joinedload on for B->A is still there somewhat unnecessary as you already loaded A, but at least the ORM will ignore those columns on the second query:

class A(Base):
    __tablename__ = 'a'

    id = Column('id', Integer, autoincrement=True, primary_key=True)
    name = Column('name', String(255))
    bs = relationship('B', back_populates='a', lazy='selectin')

    def __repr__(self) -> str:
        return f'A({self.id},{self.name},{self.bs})'

class B(Base):
    __tablename__ = 'b'

    id = Column('id', Integer, autoincrement=True, primary_key=True)
    a_id = Column('a', Integer, ForeignKey('a.id'))
    a = relationship('A', back_populates='bs', lazy='joined')
    name = Column('name', String(255))

    def __repr__(self) -> str:
        return f'B({self.id},{self.name},{self.a.id})'

with the second approach you get two queries but they are fairly decent:

SELECT a.id, a.name 
FROM a
SELECT b.a AS b_a, b.id AS b_id, b.name AS b_name, a_1.id AS a_1_id, a_1.name AS a_1_name 
FROM b LEFT OUTER JOIN a AS a_1 ON a_1.id = b.a 
WHERE b.a IN (%s, %s, %s)

overall i think the main idea is to use two different kinds of loading on the two sides by default.