1.10.14 Webserver : webserver launch crash without failling
See original GitHub issueApache Airflow version: 1.10.14
Environment: docker -> FROM python:3.6.8
What happened:
the webserver at launch crash but do not stop the process. So the container stay up , until I restart
What you expected to happen:
The process should fail and stop
How to reproduce it:
When I run docker-compose up -d it launch
1 container for the backend database ( postgre 13.1) 1 container for the webserver 1 container for the scheduler 1 container for the init_db ( only running airflow db init ) 1 container for the create user ( only running airflow create_user -r Admin -f admin -l admin -e None -u admin -p admin )
If I kill the webserver container and relaunch it , it work well. (So it’s apparently a case of when the database is not yet fully setup )
logs of the webserver
[SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (nextval('ab_permission_view_id_seq'), %(permission_id)s, %(view_menu_id)s) RETURNING ab_permission_view.id]
[parameters: {'permission_id': 84, 'view_menu_id': 96}]
(Background on this error at: http://sqlalche.me/e/13/gkpj)
[2021-04-09 11:12:38,582] {security.py:210} INFO - Initializing permissions for role:Viewer in the database.
Traceback (most recent call last):
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
cursor, statement, parameters, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
cursor.executemany(statement, parameters)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL: Key (permission_view_id, role_id)=(47, 3) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/venv/bin/myflow", line 8, in <module>
sys.exit(main())
File "/opt/venv/lib/python3.6/site-packages/myflow/myflow_cli.py", line 75, in main
args.func(args)
File "/opt/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 81, in wrapper
return f(*args, **kwargs)
File "/opt/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 1179, in webserver
app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 297, in cached_app
app, _ = create_app(config, session, testing)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 213, in create_app
security_manager.sync_roles()
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 487, in sync_roles
self.init_role(role, vms, perms)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 216, in init_role
self.get_session.merge(role)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2160, in merge
self._autoflush()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 540, in execute
self.dependency_processor.process_saves(uow, states)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1177, in process_saves
uowcommit, secondary_insert, secondary_update, secondary_delete
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1239, in _run_crud
connection.execute(statement, secondary_insert)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
cursor, statement, parameters, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
cursor.executemany(statement, parameters)
sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL: Key (permission_view_id, role_id)=(47, 3) already exists.
[SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s)]
[parameters: ({'permission_view_id': 47, 'role_id': 3}, {'permission_view_id': 101, 'role_id': 3}, {'permission_view_id': 57, 'role_id': 3}, {'permission_view_id': 70, 'role_id': 3}, {'permission_view_id': 207, 'role_id': 3}, {'permission_view_id': 167, 'role_id': 3}, {'permission_view_id': 96, 'role_id': 3}, {'permission_view_id': 68, 'role_id': 3} ... displaying 10 of 41 total bound parameter sets ... {'permission_view_id': 161, 'role_id': 3}, {'permission_view_id': 85, 'role_id': 3})]
(Background on this error at: http://sqlalche.me/e/13/gkpj)
maybe this is a problem already fix with airflow 2.0.0 , in that case I’m sorry for opening this issue 😃
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (3 by maintainers)
Top GitHub Comments
https://github.com/apache/airflow/pull/13470
Duplicate of #13470 indeed