wagtail 4.0rc1 migration ValueError: Could not find object DocumentChooserBlock

Issue Summary

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, core, custom_documents, custom_images, custom_users, home, ...,
  No migrations to apply.
Traceback (most recent call last):
  File "django/./manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File ".venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File ".venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File ".venv/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File ".venv/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File ".venv/lib/python3.10/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File ".venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 227, in handle
    changes = autodetector.changes(graph=executor.loader.graph)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 41, in changes
    changes = self._detect_changes(convert_apps, graph)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 184, in _detect_changes
    self.generate_altered_fields()
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 964, in generate_altered_fields
    old_field_dec = self.deep_deconstruct(old_field)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 78, in deep_deconstruct
    [self.deep_deconstruct(value) for value in args],
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 78, in <listcomp>
    [self.deep_deconstruct(value) for value in args],
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 54, in deep_deconstruct
    return [self.deep_deconstruct(value) for value in obj]
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 54, in <listcomp>
    return [self.deep_deconstruct(value) for value in obj]
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 56, in deep_deconstruct
    return tuple(self.deep_deconstruct(value) for value in obj)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 56, in <genexpr>
    return tuple(self.deep_deconstruct(value) for value in obj)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 78, in deep_deconstruct
    [self.deep_deconstruct(value) for value in args],
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 78, in <listcomp>
    [self.deep_deconstruct(value) for value in args],
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 54, in deep_deconstruct
    return [self.deep_deconstruct(value) for value in obj]
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 54, in <listcomp>
    return [self.deep_deconstruct(value) for value in obj]
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 56, in deep_deconstruct
    return tuple(self.deep_deconstruct(value) for value in obj)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 56, in <genexpr>
    return tuple(self.deep_deconstruct(value) for value in obj)
  File ".venv/lib/python3.10/site-packages/django/db/migrations/autodetector.py", line 71, in deep_deconstruct
    deconstructed = obj.deconstruct()
  File ".venv/lib/python3.10/site-packages/wagtail/blocks/base.py", line 344, in deconstruct
    raise ValueError(
ValueError: Could not find object CustomDocumentChooserBlock in wagtail.blocks.base.
Please note that you cannot serialize things like inner classes. Please move the object into the main module body to use migrations.

I’m using a custom document model (WAGTAILDOCS_DOCUMENT_MODEL = "custom_documents.CustomDocument") so I removed WAGTAILDOCS_DOCUMENT_MODEL setting it see if the error persisted and when running ./manage.py migrate the ValueError changes to:

ValueError: Could not find object DocumentChooserBlock in wagtail.blocks.base.

CustomDocumentChooserBlock -> DocumentChooserBlock

Steps to Reproduce

1. run ./manage.py migrate or ./manage.py makemigrations

• I couldn’t reproduce this on a fresh install of wagtail 4.0rc1 however I have been unable to find a cause for this error in my project so far.

Technical details

• Python version: 3.10.5
• Django version: 3.2.15
• Wagtail version: 4.0rc1

[gasman]

PR created: #9004

This reminds me of something that has surprised me in the past: when blocks that inherit from from wagtail.blocks.ChooserBlock are serialized for migrations, they don’t get serialized as “plain instances” as documented here, like, say, blocks that inherit from ChoiceBlock do. Presumably this is because ChooserBlocks require some custom code implementation in a way that other block types do not?

Yep, that’s correct. Whenever a custom Python object (such as a block object) appears in a model definition, the result of calling deconstruct() on that object will be “baked in” to the relevant migration files, which means that the creator of that object is effectively making a commitment that its definition will remain in place forever (or else people would have to go back and fix up historical migration files). This sort of commitment is reasonable for “extension code” (i.e. the sort of thing you might make into a Wagtail package), but not for “application code” (since the whole point of migrations is that developers should be able to add and remove things freely).

In the case of StructBlock and StreamBlock, there’s a standardised pattern for subclassing them in a data-driven way within application code, and the base StructBlock / StreamBlock classes know enough about that pattern to be able to deconstruct them back into basic instances (providing they’re not doing any backwards-incompatible code-level customisations alongside specifying child blocks), so the application developer isn’t bound by that commitment. On the other hand, subclasses of ChooserBlock are liable to differ from the base class in ways that the base class can’t know about, so there’s no way to build an equivalent block object inside the migration without importing the subclass. And that’s fine, because subclassing ChooserBlock is within the realm of “extension code”, and people doing that are expected to commit to keeping those definitions around (or come up with their own deconstruct() logic to avoid that).

[danhayden]

I have also tested on the latest commits and can confirm the issue persists when using a different name for the custom document model.