question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

1.10.14 Webserver : webserver launch crash without failling

See original GitHub issue

Apache Airflow version: 1.10.14

Environment: docker -> FROM python:3.6.8

What happened:

the webserver at launch crash but do not stop the process. So the container stay up , until I restart

What you expected to happen:

The process should fail and stop

How to reproduce it:

When I run docker-compose up -d it launch

1 container for the backend database ( postgre 13.1) 1 container for the webserver 1 container for the scheduler 1 container for the init_db ( only running airflow db init ) 1 container for the create user ( only running airflow create_user -r Admin -f admin -l admin -e None -u admin -p admin )

If I kill the webserver container and relaunch it , it work well. (So it’s apparently a case of when the database is not yet fully setup )

logs of the webserver


[SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (nextval('ab_permission_view_id_seq'), %(permission_id)s, %(view_menu_id)s) RETURNING ab_permission_view.id]
[parameters: {'permission_id': 84, 'view_menu_id': 96}]
(Background on this error at: http://sqlalche.me/e/13/gkpj)
[2021-04-09 11:12:38,582] {security.py:210} INFO - Initializing permissions for role:Viewer in the database.
Traceback (most recent call last):
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
    cursor, statement, parameters, context
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
    cursor.executemany(statement, parameters)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL:  Key (permission_view_id, role_id)=(47, 3) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/venv/bin/myflow", line 8, in <module>
    sys.exit(main())
  File "/opt/venv/lib/python3.6/site-packages/myflow/myflow_cli.py", line 75, in main
    args.func(args)
  File "/opt/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 81, in wrapper
    return f(*args, **kwargs)
  File "/opt/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 1179, in webserver
    app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
  File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 297, in cached_app
    app, _ = create_app(config, session, testing)
  File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 213, in create_app
    security_manager.sync_roles()
  File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 487, in sync_roles
    self.init_role(role, vms, perms)
  File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 216, in init_role
    self.get_session.merge(role)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2160, in merge
    self._autoflush()
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
    util.raise_(e, with_traceback=sys.exc_info()[2])
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
    self.flush()
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
    self._flush(objects)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
    transaction.rollback(_capture_exception=True)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    with_traceback=exc_tb,
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
    flush_context.execute()
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
    rec.execute(self)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 540, in execute
    self.dependency_processor.process_saves(uow, states)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1177, in process_saves
    uowcommit, secondary_insert, secondary_update, secondary_delete
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1239, in _run_crud
    connection.execute(statement, secondary_insert)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
    distilled_params,
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
    e, statement, parameters, cursor, context
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
    cursor, statement, parameters, context
  File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
    cursor.executemany(statement, parameters)
sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL:  Key (permission_view_id, role_id)=(47, 3) already exists.

[SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s)]
[parameters: ({'permission_view_id': 47, 'role_id': 3}, {'permission_view_id': 101, 'role_id': 3}, {'permission_view_id': 57, 'role_id': 3}, {'permission_view_id': 70, 'role_id': 3}, {'permission_view_id': 207, 'role_id': 3}, {'permission_view_id': 167, 'role_id': 3}, {'permission_view_id': 96, 'role_id': 3}, {'permission_view_id': 68, 'role_id': 3}  ... displaying 10 of 41 total bound parameter sets ...  {'permission_view_id': 161, 'role_id': 3}, {'permission_view_id': 85, 'role_id': 3})]
(Background on this error at: http://sqlalche.me/e/13/gkpj)

maybe this is a problem already fix with airflow 2.0.0 , in that case I’m sorry for opening this issue 😃

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

0reactions
potiukcommented, Jul 15, 2021

Duplicate of #13470 indeed

Read more comments on GitHub >

github_iconTop Results From Across the Web

our nginx server is stopping/crashing on its own
You have two problems: a) your nginx crashes (and the part of the log you provided doesn't show why). b) nginx can't start...
Read more >
Changelog — Airflow Documentation
Fix airflow-webserver startup errors when using Kerberos Auth (#10047) ... Add AirflowFailException to fail without any retry (#7133).
Read more >
Paper Server is crashing without error logs anywhere (1.18)
Hetzner cloud host with pterodactyl panel. Startup Command: java -Xms128M -Xmx10240M -Dterminal.jline=false -Dterminal.ansi=true -jar server.jar.
Read more >
Can the pm2 node module restarts the app after crash ...
Yes, it does this by default. For more information see Restart strategies. If the app repeatedly fails to start over a short period...
Read more >
Known issues | Cloud Composer
3 prevents logs from showing in the Airflow web server. Upgrade to version 1.10.4 (or later) to fix this issue. GKE Workload Identity...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found