Enable non-transactional migrations

For DB safety reasons, sometimes it is necessary to add an index concurrently. With the CONCURRENTLY option:

PostgreSQL will build the index without taking any locks that prevent concurrent inserts, updates, or deletes on the table; whereas a standard index build locks out writes (but not reads) on the table until it’s done.

But this parameter cannot be used in migrations because Hasura runs the migration in a transaction when applying. But concurrent index creation cannot be wrapped in a transaction block:

Another difference is that a regular CREATE INDEX command can be performed within a transaction block, but CREATE INDEX CONCURRENTLY cannot.

There don’t seem to be any workarounds or escape hatches in this case, beyond not tracking the migration at all and running directly on the db instance.

Some solutions might be to include an option in the apply api, or encoding not wrapping a transaction in the yml for the migration file itself.

Original request in Discord: https://discordapp.com/channels/407792526867693568/535727661167673364/626175223724769295

Postgres documentation: https://www.postgresql.org/docs/9.1/sql-createindex.html

[haphut]

@avimoondra Thanks for bringing this up. Could you clarify the title of the issue, please? Maybe something like “Enable non-transactional migrations”.

Without the ability to CREATE INDEX CONCURRENTLY, we will have to stop using Hasura for migrations. The feature seems otherwise clean so it’s a bummer to have to switch.

[creatorrr]

This is also needed for any DDL actions for timescaledb for creating continuous aggregates etc because it uses two separate transactions internally. For now, a workaround is to manually apply the up.sql file and then running hasura migrate apply --up 1 --skip-execution to mark the migration applied in the tracker.

But it’d be really nice to have a CLI flag like --dangerous-manual-transaction that tells the engine to skip creating a transaction automatically and trust that the person running apply is taking care of the transactions manually.