question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error in render_revert_entity when used with asyncpg engine

See original GitHub issue

First of all, thank you for the great library, it is so much more fun than writing (and tracking) function definitions in alembic migration files!

I am using alembic_utils with asyncio sqlalchemy:

alembic==1.7.4
alembic-utils==0.7.3
SQLAlchemy==1.4.25

New functions are generated successfully, but when a function definition has been changed, alembic-utils fails with the following stack trace:

alembic revision -m 'test' --autogenerate
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.ddl.postgresql] Detected sequence named 'projects_id_seq' as owned by integer column 'projects(id)', assuming SERIAL and omitting
INFO  [alembic_utils.depends] Resolving entities with no dependencies
INFO  [alembic_utils.depends] Resolving entities with dependencies. This may take a minute
INFO  [alembic_utils.replaceable_entity] Detecting required migration op PGFunction PGFunction: public.workspace_is_visible( p_workspace_id integer, p_user_id integer )
INFO  [alembic_utils.replaceable_entity] Detecting required migration op PGFunction PGFunction: public.workspace_permission_required( p_workspace_id integer, p_user_id integer, p_permission workspace_permission )
INFO  [alembic_utils.replaceable_entity] Detecting required migration op PGFunction PGFunction: public.workspace_user_permission_list( p_workspace_id integer, p_user_id integer )
INFO  [alembic_utils.replaceable_entity] Detected ReplaceOp op for PGFunction PGFunction: public.workspace_user_permission_list( p_workspace_id integer, p_user_id integer )
INFO  [alembic_utils.replaceable_entity] Detecting required migration op PGFunction PGFunction: public.project_is_visible( p_project_id integer, p_user_id integer )
INFO  [alembic_utils.replaceable_entity] Detecting required migration op PGFunction PGFunction: public.project_permission_required( p_project_id integer, p_user_id integer, p_permission project_permission )
Traceback (most recent call last):
  File "/home/demidov/work/rndflow/api/env/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/config.py", line 588, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/config.py", line 582, in main
    self.run_cmd(cfg, options)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/config.py", line 559, in run_cmd
    fn(
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/command.py", line 234, in revision
    scripts = [script for script in revision_context.generate_scripts()]
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/command.py", line 234, in <listcomp>
    scripts = [script for script in revision_context.generate_scripts()]
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/autogenerate/api.py", line 600, in generate_scripts
    yield self._to_script(generated_revision)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/autogenerate/api.py", line 500, in _to_script
    render._render_python_into_templatevars(
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/autogenerate/render.py", line 87, in _render_python_into_templatevars
    _render_cmd_body(downgrade_ops, autogen_context)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/autogenerate/render.py", line 109, in _render_cmd_body
    lines = render_op(autogen_context, op)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic/autogenerate/render.py", line 127, in render_op
    lines = util.to_list(renderer(autogen_context, op))
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/alembic_utils/reversible_op.py", line 155, in render_revert_entity
    with engine.connect() as connection:
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/future/engine.py", line 419, in connect
    return super(Engine, self).connect()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3194, in connect
    return self._connection_cls(self, close_with_result=close_with_result)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 96, in __init__
    else engine.raw_connection()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3273, in raw_connection
    return self._wrap_pool_connect(self.pool.connect, _connection)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3240, in _wrap_pool_connect
    return fn()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect
    return _ConnectionFairy._checkout(self)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
    rec = pool._do_get()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 256, in _do_get
    return self._create_connection()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
    return _ConnectionRecord(self)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in __init__
    self.__connect()
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in __connect
    pool.logger.debug("Error on connect(): %s", e)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 748, in connect
    await_only(self.asyncpg.connect(*arg, **kw)),
  File "/home/demidov/work/rndflow/api/env/lib/python3.9/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 61, in await_only
    raise exc.MissingGreenlet(
sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called; can't call await_() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/14/xd2s)
sys:1: RuntimeWarning: coroutine 'connect' was never awaited

A temporary solution for me is to use alembic with the synchronous psycopg2 engine.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
ddemidovcommented, Nov 9, 2021

After more than 40 migrations (including extensions, functions, and triggers) I don’t see any problems with the asyncpg engine. This can be closed as far as I am concerned, thanks again!

1reaction
oliricecommented, Oct 26, 2021

There’s an update to the async_engine branch to resolve this issues with all entities except PGView (because it does control flow in SQL) if you’d like to get unblocked

Read more comments on GitHub >

github_iconTop Results From Across the Web

Asyncpg with sqlalchemy cannot perform operations ... - GitHub
Describe the bug I'm using the new async engine with asyncpg (postgresql), for a discord bot, that is running with asyncio.
Read more >
python - SQLAlchemy with asyncpg crashing with error ...
The error disappeared after adding poolclass=NullPool to create_async_engine, so here's what engine creation looks like now:
Read more >
asyncpg Documentation
asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python's asyncio framework. asyncpg requires Python 3.7 or later ......
Read more >
Workflow Orchestration - Prefect Community
However if prefect has an error it will skip some tasks. is it possible ... When it fails the orion log shows a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found