What's the recommended (painless) way for setting up data for unit testing Page-derived models?

Wagtail uses a fixture to set up its test data, which is great for core code which can depend on a set number of Page-derived models for testing. I’ve tried using fixtures to generate data for a testserver but I find that they’re very brittle and hard to work with. Getting a fixture which will load correctly into loaddata is already a pain because of dynamically-generated fields like contenttypes, so my fixtures are created with something like:

python manage.py dumpdata --exclude contenttypes --exclude auth.Permission --exclude sessions --exclude admin --natural --format=yaml --indent=4 > app/fixtures/test-fixture.yaml

Whenever adding new models or fields, however, those new contenttype entries throw everything off and old fixtures become invalid to loaddata. You’d have to create a new testing fixture every time the schema changes. Developing a Wagtail site means you’re pretty rapidly moving through schema changes while building out models, so keeping the test fixtures current becomes unworkable.

So I tabled that problem and looked into Factory Boy to create factory models to generate only the objects I need for a unit test. I can’t get that to work because factories use .create() to make objects and the Page class or one of the classes it inherits from uses a custom manager (MP_Node from treebeard?).

Am I missing something? I’m new to testing but I feel like I’ve tried some commonly recommended ideas in the Django community for managing complex testing data and the obstacles I’ve butted up against are heavily-technical edge cases that will put testing off the table for most Wagtail users.

[jeffrey-hearn]

@nkuttler Hah, nope. Sorry.

Here’s some more proof-of-concept code from before I got distracted and abandoned testing. It shows the use of Factory Boy’s fuzzy methods to generate more realistic-looking data. I needed to generate lots of mock content to fill out a website for design purposes, but it might be handy for testing, too.

import random
import math
import datetime
import socket
import struct
import os
from loremipsum import get_paragraph, get_paragraphs

from django.utils import timezone

from factory.fuzzy import *


def recent_memory_years(years=4):
    rand_float = random.uniform(0, 4)
    rand_days = int(math.trunc(365.25 * rand_float))
    rand_secs = random.randint(1, 86400)
    return timezone.now() - datetime.timedelta(days=rand_days, seconds=rand_secs)

def future_datetime_weeks(weeks=2):
    return timezone.now() + datetime.timedelta(weeks=weeks)


class FuzzyReportTitle(BaseFuzzyAttribute):
    def fuzz(self):
        lines = open( os.path.dirname(os.path.realpath(__file__)) + '/buzzwords.txt' ).read().splitlines()
        output = ''
        for x in range(random.randint(3, 6)):
            output += random.choice(lines) + ' '
        return output.strip().title()

class FuzzyIP(BaseFuzzyAttribute):
    def fuzz(self):
        return socket.inet_ntoa(struct.pack('>I', random.randint(1, 0xffffffff)))

class FuzzyURL(BaseFuzzyAttribute):
    def fuzz(self):
        return 'http://www.' + str(random.randint(1000, 9999)) + '.com'

class FuzzyCompany(BaseFuzzyAttribute):
    def fuzz(self):
        lines = open( os.path.dirname(os.path.realpath(__file__)) + '/companies.txt' ).read().splitlines()
        return random.choice(lines)

class FuzzyPhoneNumber(BaseFuzzyAttribute):
    def fuzz(self):
        return str(random.randint(100, 999)) + '-' + str(random.randint(100, 999)) + '-' + str(random.randint(1000, 9999))

class FuzzyIntroParagraph(BaseFuzzyAttribute):
    def fuzz(self):
        return get_paragraph()

class FuzzyfewParagraphs(BaseFuzzyAttribute):
    def fuzz(self):
        return "\n".join(get_paragraphs(3))

class FuzzyContent(BaseFuzzyAttribute):
    def fuzz(self):
        return "\n".join(get_paragraphs(10))

# -*- coding: utf-8 -*-

from django.test import TestCase
from django.db.models import Count

from wagtail.wagtailcore.models import Site

import factory
from factory.fuzzy import *

class CompanyIndexPageFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = CompanyIndexPage

    title = FuzzyReportTitle()
    slug = factory.fuzzy.FuzzyText(length=50)
    intro = FuzzyIntroParagraph()

class CompanyPageFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = CompanyPage

    name = FuzzyCompany()
    intro = FuzzyIntroParagraph()
    about = FuzzyfewParagraphs()
    website = FuzzyURL()
    title = factory.LazyAttribute( lambda o: o.name )
    slug = factory.fuzzy.FuzzyText(length=50)

class CompanyNotFound(Exception):
    def __init__(self, value):
        self.parameter = value
    def __str__(self):
        return repr(self.parameter)

def random_company():
    count = CompanyPage.objects.aggregate(count=Count('id'))['count']
    if count == 0:
        raise CompanyNotFound("There are no CompanyPage objects.")
    random_index = random.randint(0, count - 1)
    return CompanyPage.objects.all()[random_index]

def print_company(company):
    print ' '
    print '--- ' + company.name + ' ----------------------------'
    print '    ' + company.website
    print '    ' + company.intro
    print '    ' + company.about
    print '--------------------------------------------------------'


class CompanyTestCase(TestCase):
    def setUp(self):
        # reset any default data
        CompanyIndexPage.get_tree().all().delete()
        CompanyPage.get_tree().all().delete()
        Site.objects.all().delete()

        # generate a family tree
        self.root = CompanyIndexPage.add_root( instance=CompanyIndexPageFactory.build() )
        self.site = Site.objects.create(hostname='localhost', port=80, root_page=self.root, is_default_site=True)

        # Companies
        for x in range(10):
            self.root.add_child( instance=CompanyPageFactory.build() )

        # for company in CompanyPage.objects.all():
        #   print_company(company)

    def test_test(self):
        self.assertTrue( True )

[jeffrey-hearn]

Thank you @kaedroho @gasman .

I got hung up on an API change in Factory Boy that was in their docs but not yet in pip. After several hours of rewarding work to realize that fact 😃 , I was able to get Wagtail models working correctly with Factory Boy (below). I haven’t fully explored django-treebeard, but this sort of tree setup should suffice for alot of use cases. When I’ve had a chance to write a test suite for my project and feel like I have a handle on what all the caveats are, I’ll write up a doc detailing the usage.

# -*- coding: utf-8 -*-

import random
import datetime
import string

from django.test import TestCase

from app.models.blog import BlogPage

import factory

def random_string(length=10):
    return u''.join(random.choice(string.ascii_letters) for x in range(length))

def future_datetime_weeks(weeks=2):
    return datetime.datetime.now() + datetime.timedelta(weeks=weeks)

class BlogPageFactory(factory.django.DjangoModelFactory):
    FACTORY_FOR = 'app.BlogPage'

    title = random_string(100)
    slug = random_string(50)
    date = future_datetime_weeks()
    body = random_string(1000)

class BlogPageTestCase(TestCase):
    def setUp(self):
        BlogPage.get_tree().all().delete()
        self.root = BlogPage.add_root( instance=BlogPageFactory.build() )
        self.root.add_child( instance=BlogPageFactory.build() )

    def test_blogpage_has_content(self):
        for x in self.root.get_tree():
            print x
            print "\n\n\n"
        self.assertTrue( len(self.root.body) > 0 )