Superset db upgrade hangs for a custom alembic script

The superset db upgrade command hangs when trying to run a custom alembic script to increase the length of username column in the ab_user table. Once the upgrade runs, it creates an idle sleep connection with the database which supposedly creates a lock on the ab_user table which prevents the alter command to run. We are using a 10.1.19-MariaDB as the database which is hosted in AWS RDS.

For the first time installation, the upgrade works fine and deployment completes successfully. But when we create a new alembic script, build and restart the server the hoping the db upgrade should result

Not able to figure out how a sleep command is getting generated internally before the upgrade process. Has it to do with FAB or alembic in general?

Expected results

The db upgrade process should complete successfully

Actual results

The upgrade process hangs during

Screenshots

The list of all the process for the db. The second last row is the one which prevents the ALTER command to be executed. The time for the command is same so it seems it just go triggered during the upgrade process

The lock on ab_user table

How to reproduce the bug

1. Have an installed version of superset running with db up and running.
2. Write a custom alembic migrate script to increase the username length
3. Build the project and restart the server.
4. The db upgrade process should get stuck

Environment

• superset version: 0.999.0dev
• python version: 3.7.2
• node.js version: 10.15.0
• npm version: 6.9.0

Additional context

Generated alembic script

revision = 'c088a7201375'
down_revision = 'c82ee8a39623'

from alembic import op
import sqlalchemy as sa


def upgrade():
    with op.batch_alter_table('ab_user') as batch_op:
        batch_op.alter_column(
            'username',
            nullable=False,
            type_=sa.String(length=256),
            existing_type=sa.String(length=64),
            existing_server_default=None, 
            existing_nullable=False)


def downgrade():
    with op.batch_alter_table('ab_user') as batch_op:
        batch_op.alter_column(
            'username',
            nullable=False,
            type_=sa.String(length=64),
            existing_type=sa.String(length=256),
            existing_server_default=None, 
            existing_nullable=False)

Below are the logs from the console when the db gets stuck

xxx-superset             | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
xxx-superset             | INFO  [alembic.runtime.migration] Will assume transactional DDL.
xxx-superset             | INFO  [alembic.runtime.migration] Running upgrade c82ee8a39623 -> c088a7201375, increase username length

The upgrade process gets stuck for ~ 8 hours as the sleep command times out after that time as per the default db configuration time for time_wait_timeout and wait_timeout of 28800 seconds. After the time out, the ALTER command gets executed. Also, if we manually KILL the thread, the ALTER gets executed and the upgrade finishes successfully.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.96. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.