properly reset databases on tests teardown

Currently multiple databases are supported by creating instance of Migrator for each database, but only default database is cleaned at the end of test - https://github.com/wemake-services/django-test-migrations/blob/master/django_test_migrations/migrator.py#L77

Also using migrate management command is not the best idea when it comes to performance, because there is no need to apply all unapplied migrations just before the next test, which should start with clean state. Django’s TestCase uses flush management command to clean databases after test.

We need to reuse Django’s code responsible for cleaning database on test teardown (or establish our own way to do that) and fix Migrator.reset() method to clean Migrator.database instead of default one.

[skarzi]

@asfaltboy I have digged into Django’s migrations deeper and had some answers for your problem, but also many new concerns that I’d like to discuss.

Note that I tried testing #46 locally and I’m having issues with dependency migrations not being applied (while they are being applied in current 0.2.0 ).

This issue is related to how Django built migrations’ plan. In 0.2.0 migration’s test setup/assert/teardown flow looks like below:

(NOTE: in both cases, before running tests, django calls migrate command to migrate all forwards).

1. Use migration executor to migrate backward to migrate_from - migrating backward to migrate_from unapplies only migrations that are higher in migrations plan than migrate_from
2. Create some db models instances to test migrate_to on them
3. Migrate forward to migrate_to
4. Do assertions
5. Call migrate command to migrate all forwards

Flow implemented in #46 looks like below:

1. Drop all model’s tables
2. Flush django_migrations table
3. Migrate forward to migrate_from - here migrations are applied on clean database and everything goes forward, so only migrations that are dependencies of migrate_from will be applied
4. Create some db models instances to test migrate_to on them
5. Migrate forward to migrate_to
6. Do assertions
7. Call migrate command to migrate all forwards

Let’s imagine we have 2 apps: some_app and other_app. some_app has following migrations:

• 0009_do_some_stuff - all migrations from initial to this one are not dependent in anyway on other_app
• 0010_do_some_stuff_dependent_on_other_app - this is the first migration that depends on other_app

other_app is self-sufficient when it comes to migrations (like e.g. django.contrib.sites) and it’s much lower in migrations plan than some_app.

Then let’s imagine we are testing migration 0010, so migrate_from is 0009 and migrate_to is 0010. Let’s focus on preparation step where some testing db models instances are created.

In 0.2.0 we will have access to models from other_app, because we are migrating backward and only migrations higher in the plan than migrate_before will be unapplied But in #46 we won’t be able to access other_app models, because we are migrating only forward to migrate_from which is not dependent on other_app.

The problem is more trickier than I thought 😦 Solution from 0.2.0 is not ideal because it’s not similar to how we apply migrations on production, however it’s hard to implement production like solution for migration’s test flow.

Yesterday I was experimenting with creating migrations plan manually starting from one of:

• plan generated for migrate_to targets
• complete plan generated for clean db

and then truncating such plan on migrate_before migrations, to make it more like on production (avoid migrating backwards) and preserve behaviour from 0.2.0. It’s quite “magical”/complicated solution, however I can clean it and push (but give me few days) so we can discuss it.

[asfaltboy]

Hi @skarzi thanks for investing much time into this, this is indeed a tough issue to solve.

I was thinking about this a bit more, and it seems that, typically, the reason data migrations fail to be re-applied in teardown is due to “primary key sequence” mismatch. I see that in your PR #46 you implement get_django_migrations_table_sequences to keep track of the ids … do you think it’s possible to simplify this by using django.test.TransactionTestCase.reset_sequences (also supported in pytest-django)?

Note that I tried testing #46 locally and I’m having issues with dependency migrations not being applied (while they are being applied in current 0.2.0 ). I was not able to overcome this by specifying all the related apps migrations (I gave up mid-way), but even if I did it would not be ideal. How come this is not the same behavior as on 0.2.0, I thought #46 only fixes the teardown?