Django Error when using multiple fixtures and factory.Iterator

We have several models with ForeignKey.

These models have factories using factory.Iterator on the same tables (Ex: Countries.objects.all(), Currencies.objects.all(), etc.) Ex:

• CountryID = factory.Iterator(models.Countries.objects.all())
• CurrencyID = factory.Iterator(models.Currencies.objects.all())

We have multiple django test cases using different fixtures with different data for these tables (Ex: A fixture with the country “Afghanistan”, another fixture with “Canada” and “USA”).

With SQLite it gives this kind of error:

<AppName>.models.DoesNotExist: Countries matching query does not exist.

With MySQL it gives this kind of error: django.db.utils.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_<AppName>`.`SomeModel`, CONSTRAINT `SomeModel_CurrencyID_2734797b_fk_Currencies_id` FOREIGN KEY (`CurrencyID`) REFERENCES `Currencies` (`id`))')

Tested with factory-boy 2.7.0 and 2.8.1 with django 1.9.

When many tests are ran the bug can occur. When running the tests individually they succeed.

I assume there is a problem with the iterator when unloading and loading fixtures on these table. The iterator may not be reinitialized between tests cases.

Workarounds:

1. Use the same data for these tables (either same fixture or same data in different fixtures)
2. Replace the iterator with a lamda that will randomly select a row in the table
3. Replace the iterator with a subfactory (but does not work well with some models)

[jeffwidman]

Thanks for the bug report. Afraid I’m not very familiar with how the Django ORM resolves foreign keys, so I’ll let @rbarrois handle this one.

[msimav]

We had a similar issue. After debugging we see that the error is caused by QuerySet’s stale cache when it is used with either factory.Iterator or factory.fuzzy.FuzzyChoice.

• Django QuerySets will cache data once they are evaluated
• factory.Iterator won’t evaluate the QuerySet until it is used for the first time (which is useful to prevent other bugs)
• In the setUp method, some test data is created that will be removed once the suite is run
  • In the first time, QuerySet will be cached with the data that is created for that test case
  • In the following cases, the cached data is used but it no longer exists in the db
    • That causes an integrity error
• Resetting the QuerySet is possible but factory.Iterator wraps it with another layer which makes it impossible

Minimal Example

# models.py
from django.db import models


class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField("date published")


class Choice(models.Model):
    question = models.ForeignKey(Question, on_delete=models.CASCADE)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)


# factories.py
import factory
from datetime import datetime


class QuestionFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Question

    question_text = factory.Faker("sentence")
    pub_date = factory.Faker(
        "date_between_dates",
        date_start=datetime.date(2021, 1, 1),
        date_end=datetime.date(2021, 12, 31),
    )


class ChoiceFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Choice

    question = factory.Iterator(Question.objects.all())
    choice_text = factory.Faker("sentence")
    votes = factory.fuzzy.FuzzyInteger(0, 100)


# tests.py
from django.test import TestCase


class TestCase1(TestCase):
    def setUp(self):
        QuestionFactory()
        ChoiceFactory()


class TestCase2(TestCase):
    def setUp(self):
        QuestionFactory.create_batch(5)
        ChoiceFactory.create_batch(20)

Running each test case separately (or in parallel) doesn’t trigger an error but running them sequentially does.

Our Workaround

We created a new one that checks if the model instance is stale and resets the Queryset if so.

class QuerysetIterator(BaseDeclaration):
    """Fill this value using the values returned by a queryset.

    Attributes:
        queryset (Queryset): the queryset whose value should be used.
        getter (callable or None): a function to parse returned values
    """

    def __init__(self, queryset, cycle=True, getter=None):
        super().__init__()
        self.queryset = queryset
        self.cycle = cycle
        self.getter = getter
        self.iterator = None

    def evaluate(self, instance, step, extra):
        if self.iterator is None:
            self.reset()

        try:
            value = next(iter(self.iterator))
        except StopIteration:
            if self.cycle:
                self.reset()
                value = next(iter(self.iterator))
            else:
                raise StopIteration

        try:
            value.refresh_from_db()
        except ObjectDoesNotExist:
            self.reset()
            value = next(iter(self.iterator))

        if self.getter is None:
            return value
        return self.getter(value)

    def reset(self):
        """Reset the internal iterator."""
        self.iterator = self.queryset.all()

I’d like to create a PR if the solution makes sense.