question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

airflow db upgrade failed, no generic 'DROP CONSTRAINT' in MySQL

See original GitHub issue

Apache Airflow version: 2.0.0

Kubernetes version (if you are using kubernetes) (use kubectl version): N/A

Environment: docker swarm

  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release): Ubuntu 18.04.5 LTS (Bionic Beaver)
  • Kernel (e.g. uname -a):
  • Install tools: mysql-client=5.7
  • Others:

What happened:

When I try to upgrade airflow db with this following command:

airflow db upgrade

Something wrong happened :

initdb_1  | DB: mysql://**:***@x.x.x.x:****/mydb
initdb_1  | [2020-12-21 15:32:09,044] {db.py:678} INFO - Creating tables
initdb_1  | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
initdb_1  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
initdb_1  | INFO  [alembic.runtime.migration] Running upgrade 03afc6b6f902 -> cf5dc11e79ad, drop_user_and_chart
initdb_1  | Traceback (most recent call last):
initdb_1  |   File "/usr/local/bin/airflow", line 8, in <module>
initdb_1  |     sys.exit(main())
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
initdb_1  |     args.func(args)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
initdb_1  |     return func(*args, **kwargs)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 89, in wrapper
initdb_1  |     return f(*args, **kwargs)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb
initdb_1  |     db.upgradedb()
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 688, in upgradedb
initdb_1  |     command.upgrade(config, 'heads')
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
initdb_1  |     script.run_env()
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
initdb_1  |     util.load_python_file(self.dir, "env.py")
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
initdb_1  |     module = load_module_py(module_id, path)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
initdb_1  |     spec.loader.exec_module(module)
initdb_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
initdb_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 108, in <module>
initdb_1  |     run_migrations_online()
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
initdb_1  |     context.run_migrations()
initdb_1  |   File "<string>", line 8, in run_migrations
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
initdb_1  |     self.get_context().run_migrations(**kw)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 522, in run_migrations
initdb_1  |     step.migration_fn(**kw)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/migrations/versions/cf5dc11e79ad_drop_user_and_chart.py", line 49, in upgrade
initdb_1  |     op.drop_constraint('known_event_user_id_fkey', 'known_event')
initdb_1  |   File "<string>", line 8, in drop_constraint
initdb_1  |   File "<string>", line 3, in drop_constraint
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/operations/ops.py", line 159, in drop_constraint
initdb_1  |     return operations.invoke(op)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/operations/base.py", line 373, in invoke
initdb_1  |     return fn(self, operation)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/operations/toimpl.py", line 163, in drop_constraint
initdb_1  |     schema=operation.schema,
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/ddl/mysql.py", line 113, in drop_constraint
initdb_1  |     super(MySQLImpl, self).drop_constraint(const)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/ddl/impl.py", line 248, in drop_constraint
initdb_1  |     self._exec(schema.DropConstraint(const))
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/ddl/impl.py", line 141, in _exec
initdb_1  |     return conn.execute(construct, *multiparams, **params)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
initdb_1  |     return meth(self, multiparams, params)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
initdb_1  |     return connection._execute_ddl(self, multiparams, params)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1066, in _execute_ddl
initdb_1  |     else None,
initdb_1  |   File "<string>", line 1, in <lambda>
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 481, in compile
initdb_1  |     return self._compiler(dialect, bind=bind, **kw)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 29, in _compiler
initdb_1  |     return dialect.ddl_compiler(dialect, self, **kw)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 322, in __init__
initdb_1  |     self.string = self.process(self.statement, **compile_kwargs)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 352, in process
initdb_1  |     return obj._compiler_dispatch(self, **kwargs)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/compiler.py", line 441, in <lambda>
initdb_1  |     lambda *arg, **kw: existing(*arg, **kw),
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/compiler.py", line 486, in __call__
initdb_1  |     return fn(element, compiler, **kw)
initdb_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/ddl/mysql.py", line 394, in _mysql_drop_constraint
initdb_1  |     "No generic 'DROP CONSTRAINT' in MySQL - "
initdb_1  | NotImplementedError: No generic 'DROP CONSTRAINT' in MySQL - please specify constraint type

What you expected to happen:

database successfully upgraded (migration rules applied without errors)

How to reproduce it:

Upgrade existing database with mysql dialect from 1.10.14 to 2.0.0

Anything else we need to know:

every times

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:13 (13 by maintainers)

github_iconTop GitHub Comments

1reaction
rkhaled0commented, Dec 21, 2020

@potiuk I can create a PR for that if you approve these changes.

0reactions
potiukcommented, Jan 5, 2021

Nice! Thanks!

Fixed by #13239

Read more comments on GitHub >

github_iconTop Results From Across the Web

[GitHub] [airflow] potiuk commented on issue #13222: airflow db ...
It looks like MySQL SQLalchemy client that you use might simply not recognize ... else: raise NotImplementedError( "No generic 'DROP CONSTRAINT' in MySQL...
Read more >
Upgrading Airflow to a newer version - Apache Airflow
Newer Airflow versions can contain database migrations so you must run airflow db upgrade to upgrade your database with the schema changes in...
Read more >
Release Notes — Airflow Documentation
Bugfix: no generic DROP CONSTRAINT in MySQL during airflow db upgrade (#13239). Bugfix: Sync Access Control defined in DAGs when running sync-perm (#13377)....
Read more >
Best Practices - Apache Airflow
Specifically you should not run any database access, heavy computations and ... It's primary purpose is to fail a DAG Run when any...
Read more >
Set up a Database Backend - Apache Airflow
Despite big similarities between MariaDB and MySQL, we DO NOT support MariaDB as a ... you need to manually upgrade SQLite to use...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found