Tests failing only when using django-nose and Django 2.X

EDIT: See update in second post, I’ve shrunk the example code down quite a bit and I’m still having this issue.

Double edit: Fix proposed and pull request submitted, waiting on confirmation that the fix is valid.

I ran into this while trying to upgrade from Django 1.11 to 2.2

Previously on Django 1.11 I had django-nose installed and all the tests ran without issue. When I tried to upgrade to Django 2, every test that touches the database in any way now throws the error:

======================================================================
ERROR: Test the index page
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/tests.py", line 115, in test_index
    response = get_and_check_page(self, 'huntserver:index', 200)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/tests.py", line 42, in get_and_check_page
    response = test.client.get(reverse(page, kwargs=args))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 517, in get
    response = super().get(path, data=data, secure=secure, **extra)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 332, in get
    return self.generic('GET', path, secure=secure, **r)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 404, in generic
    return self.request(**r)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 485, in request
    raise exc_value
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/info_views.py", line 17, in index
    curr_hunt = Hunt.objects.get(is_current_hunt=True)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 397, in get
    num = len(clone)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 254, in __len__
    self._fetch_all()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1179, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
    cursor.close()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 85, in close
    while self.nextset():
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 173, in nextset
    nr = db.next_result()
_mysql_exceptions.OperationalError: (2006, '')
-------------------- >> begin captured logging << --------------------
django.request: ERROR: Internal Server Error: /
Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1063, in execute_sql
    cursor.execute(sql, params)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 80, in _execute
    self.db.validate_no_broken_transaction()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/base/base.py", line 437, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/info_views.py", line 17, in index
    curr_hunt = Hunt.objects.get(is_current_hunt=True)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 397, in get
    num = len(clone)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 254, in __len__
    self._fetch_all()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1179, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
    cursor.close()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 85, in close
    while self.nextset():
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 173, in nextset
    nr = db.next_result()
_mysql_exceptions.OperationalError: (2006, '')
--------------------- >> end captured logging << ---------------------

This only happens on Django 2.X, and only when django-nose’s test runner is enabled. Removing the line

TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'

from settings.py fixes the issue.

These errors also don’t seem to come up at all outside of unit testing (which I guess makes sense if the issue is somehow related to django-nose).

Here is an example test from the test.py file:

from django.test import TestCase
from django.urls import reverse
from huntserver import models

def get_and_check_page(test, page, code, args={}):
    response = test.client.get(reverse(page, kwargs=args))
    test.assertEqual(response.status_code, code)
    return response

class InfoTests(TestCase):
    fixtures = ["basic_hunt"]

    def test_index(self):
        "Test the index page"
        response = get_and_check_page(self, 'huntserver:index', 200)
        self.assertTrue(isinstance(response.context['curr_hunt'], models.Hunt))

I’m seeing these issues locally locally on Debian 9 (Stretch), but it also seems to fail in the same way on travis-ci (Which appears to be ubuntu 16.04): https://travis-ci.org/dlareau/puzzlehunt_server/builds/636368209

mysql --version:

mysql  Ver 15.1 Distrib 10.1.41-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

pip freeze: (Currently at Django 2.0, but same error on 2.1 and 2.2)

alabaster==0.7.12
Babel==2.8.0
bootstrap-admin==0.4.3
certifi==2019.11.28
chardet==3.0.4
coverage==4.5.1
decorator==4.1.2
Django==2.0
django-crispy-forms==1.8.1
django-debug-toolbar==1.8
django-nose==1.4.6
django-ratelimit==1.1.0
docutils==0.16
idna==2.8
imagesize==1.2.0
Jinja2==2.10.3
MarkupSafe==1.1.1
mysqlclient==1.3.14
networkx==2.0
nose==1.3.7
packaging==20.0
Pygments==2.5.2
pyparsing==2.4.6
PyPDF2==1.26.0
python-dateutil==2.6.1
pytz==2019.3
requests==2.22.0
six==1.11.0
snowballstemmer==2.0.0
Sphinx==2.3.1
sphinx-rtd-theme==0.4.3
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sqlparse==0.2.3
urllib3==1.25.7

Database settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'puzzlehunt_db',
        'HOST': 'localhost',
        'PORT': '3306',
        'USER': 'hunt',
        'PASSWORD': 'NotMyRealMysqlPassword',
        'OPTIONS': {'charset': 'utf8mb4'},
    }
}

I’ve looked into each of the listed errors (the Transaction issue/the “OperationalError”). A lot of what I found searching keywords wasn’t applicable, though I tried many of the solutions anyway to no avail.

If it is relevant, the source code for my whole project can be found here: https://github.com/dlareau/puzzlehunt_server/tree/development

Let me know if there is any other info I can provide.

Thanks for any help!

[jwhitlock]

Thanks @dlareau for the bug, analysis, and fix! I’ve filed #314 for the follow-on issue of 1) testing fixture loading in this project, and 2) evaluating if the MySQL fixture loading workarounds are still needed.

I’ve merged #308, it ~will be~ is in release 1.4.7.

[dlareau]

Pull request #308 made, all functional tests appear to pass.