RuntimeError: dictionary changed size during iteration

Describe the bug Not sure Ormar will be able to do anything.

To Reproduce

Upon running a query, there’s some Pydantic validation happening, and it ends up with a runtime error when deep-copying data. It seems that it is due to an element (of the model being deep-copied) being a submodel, and that upon accessing it, or another attribute of the parent model, it is loaded, changing the dictionary size somehow.

Here’s the query:

packages = await Package.objects.select_related(["library", "tickets__ticket"]).exclude(version__contains=".x").all()

That query triggers the runtime error mentioned above. If I either remove tickets__ticket, or go down to the submodel with tickets__ticket__team, the error disappears.

packages = await Package.objects.select_related(["library"]).exclude(version__contains=".x").all()  # ok
packages = await Package.objects.select_related(["library", "tickets__ticket__team"]).exclude(version__contains=".x").all()  # ok

Traceback:

$ python sara.py 
Traceback (most recent call last):
  File "sara.py", line 26, in <module>
    asyncio.run(amain())
  File "/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "sara.py", line 9, in amain
    monitoring_ticket_id = await suggest_monitoring()
  File "/home/user/dev/project/src/project/sara/monitoring.py", line 114, in suggest
    libraries = await suggest_monitoring_libraries()
  File "/home/user/dev/project/src/project/suggest.py", line 150, in suggest_monitoring_libraries
    await Package.objects.select_related(["library", "tickets__ticket"]).exclude(version__contains=".x").all()
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/queryset/queryset.py", line 1053, in all
    result_rows = self._process_query_result_rows(rows)
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/queryset/queryset.py", line 184, in _process_query_result_rows
    result_rows = [
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/queryset/queryset.py", line 185, in <listcomp>
    self.model.from_row(
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/models/model_row.py", line 84, in from_row
    item = cls._populate_nested_models_from_row(
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/models/model_row.py", line 203, in _populate_nested_models_from_row
    child = model_cls.from_row(
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/models/model_row.py", line 104, in from_row
    instance = cast("Model", cls(**item))
  File "/home/user/dev/project/__pypackages__/3.8/lib/ormar/models/newbasemodel.py", line 140, in __init__
    values, fields_set, validation_error = pydantic.validate_model(
  File "pydantic/main.py", line 1038, in pydantic.main.validate_model
  File "pydantic/fields.py", line 857, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1067, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 857, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1074, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 1121, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 313, in pydantic.class_validators._generic_validator_basic.lambda12
  File "pydantic/main.py", line 679, in pydantic.main.BaseModel.validate
  File "pydantic/main.py", line 605, in pydantic.main.BaseModel._copy_and_set_values
  File "/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/copy.py", line 229, in _deepcopy_dict
    for key, value in x.items():
RuntimeError: dictionary changed size during iteration

Expected behavior No runtime error.

Versions (please complete the following information):

• Database backend used: sqlite
• Python version: 3.8.11
• ormar version: 0.11.0
• pydantic version: 1.9.1
• if applicable fastapi version: 0.78.0

Additional context The code triggering the error did not always trigger it. It might be due to a dependency upgrade (which one? I don’t know). Did anyone encounter the same issue?

[pawamoy]

Yep, everything is working fine again 🙂 Thanks!

[collerek]

Should be fixed in 0.11.1. Please check and let me know. 😉