Migrations failure

When applying the migrations from the new django specific app the following error is thrown.

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 93, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/executor.py", line 19, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/loader.py", line 47, in __init__
    self.build_graph()
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/loader.py", line 308, in build_graph
    _reraise_missing_dependency(migration, parent, e)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/loader.py", line 291, in _reraise_missing_dependency
    raise exc
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/loader.py", line 301, in build_graph
    self.graph.add_dependency(migration, key, parent)
  File "/home/tim/.virtualenvs/api-kk/lib/python3.5/site-packages/django/db/migrations/graph.py", line 116, in add_dependency
    parent
django.db.migrations.graph.NodeNotFoundError: Migration social_django.0004_auto_20160423_0400 dependencies reference nonexistent parent node ('social_django', '0003_alter_email_max_length')

[koniiiik]

I could reproduce the same error with a pre-existing database from before the reorganization, where migrations 0001 ­– 0004 were applied with default as the appname.

After running update django_migrations set app='social_django' where app='default'; in a dbshell, showmigrations is capable of finishing successfully, but it still shows all social_django migrations as not applied (even though the first four are already applied). In this state, obviously, running migrate leads to an error, because it tries to apply all of them from the beginning, but the tables are already there.

The solution (in my situation) is to run migrate --fake social_django 0004, which adds entries to django_migrations for all already-applied migrations for appnames default and social_auth as well. So in the end, I have entries in the django_migrations table for all of the already-applied migrations for all three appnames: default, social_auth, and social_django. At this point migrations work correctly for me again.

I did some more digging, and it looks like the problem here is that the replaces attribute of a Django migration has a different meaning than what is assumed in the social_django package. This does not appear to be documented (or at least, I didn’t find it in the docs), but https://github.com/django/django/blob/f582db1296cc18e2ff49846c9be4ad999759adcc/django/db/migrations/migration.py#L39-L42 sheds a little bit of light on the problem.

In short, the way replaces is used in this package seems to be along the lines of “Consider this migration applied if any of these two original migrations has been applied before,” whereas the actual meaning as understood by Django is “This migration is a replacement for the entire sequence of these two original migrations.” So, if Django only sees one of the original migrations being applied, but not the other one, it assumes the replacement is not yet applied either. And conversely, when the replacement is being applied, all of the originals get recorded, too, because there might be some other migrations elsewhere that depend on some of the originals. Which also means that if any of the original migrations is missing in the database, the replacement cannot be considered applied either.

Not sure what the way forward would be, though. I guess removing one of the two entries in replaces in all migrations would be necessary, and also documenting that a manual migrate --fake will be required in certain situations. As far as I understand, Django does not provide any more straight-forward automatic upgrade path for situations like this (but I may be wrong).

[omab]

Closing since #25 is merged, reopen if needed. Thanks @nijel.