Illegal mix of collations (utf8mb4_unicode_ci,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) in 0075_populate_latest_revision_and_revision_object_str
See original GitHub issueIssue Summary
When migrating a django project (on a fresh db) that uses mariadb:10 with --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
, the migration 0075_populate_latest_revision_and_revision_object_str
of wagtailcore fails with the following traceback. I unfortunately failed to find out which two were the offending fields. I also couldn’t find any db settings to make the migration pass anyway
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 75, in execute
return self.cursor.execute(query, args)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1267, "Illegal mix of collations (utf8mb4_unicode_ci,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) for operation '='")
The above exception was the direct cause of the following exception:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./manage.py", line 27, in <module>
main()
File "./manage.py", line 23, in main
execute_from_command_line(sys.argv)
File "/app/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/app/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/app/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/app/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/app/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 96, in wrapped
res = handle_func(*args, **kwargs)
File "/app/.venv/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 349, in handle
post_migrate_state = executor.migrate(
File "/app/.venv/lib/python3.8/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
File "/app/.venv/lib/python3.8/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
File "/app/.venv/lib/python3.8/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
File "/app/.venv/lib/python3.8/site-packages/django/db/migrations/migration.py", line 125, in apply
operation.database_forwards(
File "/app/.venv/lib/python3.8/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
self.code(from_state.apps, schema_editor)
File "/app/.venv/lib/python3.8/site-packages/wagtail/migrations/0075_populate_latest_revision_and_revision_object_str.py", line 18, in populate_latest_revision
Page.objects.all().update(latest_revision_id=latest_revision_id)
File "/app/.venv/lib/python3.8/site-packages/django/db/models/query.py", line 1190, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/app/.venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
cursor = super().execute_sql(result_type)
File "/app/.venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
cursor.execute(sql, params)
File "/app/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/__init__.py", line 562, in execute
return real_execute(self, sql, params)
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/app/.venv/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/app/.venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 75, in execute
return self.cursor.execute(query, args)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/app/.venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1267, "Illegal mix of collations (utf8mb4_unicode_ci,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) for operation '='")
Steps to Reproduce
- Start a mariadb:10 docker container with
--character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
git clone https://github.com/meine-stadt-transparent/meine-stadt-transparent
cd meine-stadt-transparent
git checkout 7af8a4ee514c8d19fc7b54ca41ae372c8fe8eede
poetry install
# tested with poetry==1.2.2./manage.py migrate wagtailcore
Any other relevant information. For example, why do you consider this a bug and what did you expect to happen instead?
- I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: Unfortunately, I failed to reproduce this with a fresh project. I think it needs the previous migrations to cause the bug. It happens with a completely fresh database when only migrating wagtailcore though
Technical details
- Python version: Run
python --version
.python:3.8
docker container - Django version: Look in your requirements.txt, or run
pip show django | grep Version
. 4.1.2 (in poetry.lock) - Wagtail version: Look at the bottom of the Settings menu in the Wagtail admin, or run
pip show wagtail | grep Version:
. 4.0.2 (in poetry.lock) - Browser version: n/a
Issue Analytics
- State:
- Created a year ago
- Comments:7 (5 by maintainers)
Top Results From Across the Web
Illegal mix of collations (utf8mb4_unicode_ci,IMPLICIT) and ...
I am guessing you have different collations on the tables you are joining. It says you are using an illegal mix of collations...
Read more >Illegal mix of collations (utf8mb4_general_ci,IMPLICIT) and ...
I use this command fix the problem: ALTER TABLE zl_report_user CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;.
Read more >Illegal mix of collations (utf8mb4_unicode_ci,EXPLICIT) and ...
Hello guys, I create one view in MySQL and after that try use in Metabase with CUSTOM QUESTION and it is introducing the...
Read more >MySQL error: Illegal mix of collations (utf8mb4_unicode_ci ...
Hi everyone, I have a couple automations that send Telegram messages with Unicode characters: - alias: 'HASS started' trigger: platform: ...
Read more >Illegal mix of collations (utf8_general_ci,IMPLICIT ... - RO CSVI
hello, on my import I have this error: An error has occured during the import. Below the information received from the server. Illegal...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I don’t think any action is necessary in Wagtail. Wagtail is using legal Django/SQL syntax.
However it would be worth adding a note to the docs, if and when mysqlclient accepts the change. I am planning to add such a note to both Wagtail and Django docs whenever the patch is released in mysqlclient.
It may be useful to temporarily add a note to the Wagtail 4 upgrade considerations, as this behavior will affect a large portion of MySQL users.
I don’t think setting
db_collation
would help. In this case, we have two tables/columns which are both using the exact same collation. However the connection is using an entire different and incorrect collation. The CAST statement uses the connection’s collation, not the table’s/column’s. This is a documented behavior of MySQL and MariaDB. The Django ORM does not add custom collation or charset args to the CAST statement, even if you setdb_collation
on the fields.The only workaround is to not use a Cast statement in Django, or to use my patch to mysqlclient which correctly sets the collation of the connection. If you have the rare setup where your tables/columns are intentionally using different collations, then there would be no possible way to use the Cast statement in Django between those columns.
I have not been able to get a response from the maintainer of
mysqlclient
.