Add support for alembic migrations
See original GitHub issueScrolling through the issue as I was setting this up, I found the following that appear to be relevant. #43 #7 #20 and #39 .
Basically it comes down to the way that the triggers are added into the DB is via the after_create
signal in SQLAlchemy. This unfortunately only triggers the first time a table is created and it only triggers when you are using the table metadata.
Working with Alembic, Flask-Migrate or other wrappers around alembic this means that the postgres triggers are never properly added and as such nothing ever gets added to the activity table.
#46 is my first pass at attempting to resolve this problem. Its not perfect but it is a start.
Basically at this point all the SQL calls back been removed to external functions from the base VersionManager
. In addition, new functions have been added to the migrations file.
manually adding init_before_create_transaction
, init_activity_table_triggers
, rollback_create_transaction
and register_table
allow most things to function.
Basically the first migration will look something like this:
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.execute(text("CREATE EXTENSION btree_gist;"))
init_before_create_transaction(op)
op.create_table('ipam_vrfs',
sa.Column('DateCreated', sa.DateTime(), nullable=True),
sa.Column('LastModified', sa.DateTime(), nullable=True),
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('name', sa.String(), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_table('transaction',
sa.Column('id', sa.BigInteger(), nullable=False),
sa.Column('native_transaction_id', sa.BigInteger(), nullable=True),
sa.Column('issued_at', sa.DateTime(), nullable=True),
sa.Column('client_addr', postgresql.INET(), nullable=True),
sa.Column('actor_id', sa.Text(), nullable=True),
postgresql.ExcludeConstraint((sa.column('native_transaction_id'), '='), (text("tsrange(issued_at - INTERVAL '1 hour', issued_at)"), '&&'), using='gist', name='transaction_unique_native_tx_id'),
sa.PrimaryKeyConstraint('id')
)
op.create_table('activity',
sa.Column('id', sa.BigInteger(), nullable=False),
sa.Column('schema_name', sa.Text(), nullable=True),
sa.Column('table_name', sa.Text(), nullable=True),
sa.Column('relid', sa.Integer(), nullable=True),
sa.Column('issued_at', sa.DateTime(), nullable=True),
sa.Column('native_transaction_id', sa.BigInteger(), nullable=True),
sa.Column('verb', sa.Text(), nullable=True),
sa.Column('old_data', postgresql.JSONB(astext_type=sa.Text()), server_default='{}', nullable=True),
sa.Column('changed_data', postgresql.JSONB(astext_type=sa.Text()), server_default='{}', nullable=True),
sa.Column('transaction_id', sa.BigInteger(), nullable=True),
sa.ForeignKeyConstraint(['transaction_id'], ['transaction.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_activity_native_transaction_id'), 'activity', ['native_transaction_id'], unique=False)
init_activity_table_triggers(op.get_bind())
register_table(op, "ipam_vrfs", ["DateCreated", "LastModified"])
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
rollback_create_transaction(op)
op.drop_index(op.f('ix_activity_native_transaction_id'), table_name='activity')
op.drop_table('activity')
op.drop_table('transaction')
op.drop_table('ipam_vrfs')
# ### end Alembic commands ###
All future migrations will need lines like register_table(op, "ipam_vrfs", ["DateCreated", "LastModified"])
added for all tables that have had either their `excluded_columns modified or a new table for revision tracking is added.
Excluded columns are explicitly added and not read from the models to allow full history tracking of what was ignored.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:2
- Comments:6
PR has been updated to include support for alembic autogenerated migrations. To support this,
__versioned__
needed to be moved into the__table_args__['info']
dict under theversioned
key. This appeared to be an acceptable move by comments in #7.To enable alembic support in your env.py file you need to add a couple lines.
postgresql_audit.alembic.writer.process_revision_directives
needs to be passed in to thecontext.configure
call under theprocess_revision_directives
keyword arg.If you are using flask-migrate,
process_revision_directives
function can be slightly modified tothis needs superuser privileges. [1] suggests to use the following instead:
op.execute(text("CREATE EXTENSION IF NOT EXISTS btree_gist;"))
[1]https://stackoverflow.com/a/51395480/10617111