Support for PgBouncer transaction pooling

Which version and edition of Flyway are you using?

6.5.2

If this is not the latest version, can you reproduce the issue with the latest one as well?

(Many bugs are fixed in newer releases and upgrading will often resolve the issue)

Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)

Java API

Which database are you using (type & version)?

postgres (PostgreSQL) 10.13 PgBouncer 1.14.0

Which operating system are you using?

CentOS Linux release 7.8.2003

What did you do?

(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)

Connected to the PostgreSQL database via PgBouncer, using transaction pooling. Flyway.migrate() is called as part of startup.

This is part of a multi-node environment where multiple nodes start simultaneously, meaning there are multiple nodes starting Flyway from the same code-base at the same time.

What did you expect to see?

Validation of the existing migrations and performing any missing ones.

What did you see instead?

Multiple nodes appear to “lock”, waiting for a time-out and then dying with an error “Number of retries exceeded while attempting to acquire PostgreSQL advisory lock”.

Looking at the way the code works, this is expected; Flyway grabs a session lock as part of its work (https://github.com/flyway/flyway/blob/master/flyway-core/src/main/java/org/flywaydb/core/internal/database/postgresql/PostgreSQLAdvisoryLockTemplate.java#L106-L116) but PgBouncer is set up to do transaction pooling, so multiple nodes may (and do, in this case) receive the same database session for separate transactions. This then means that the session lock can’t be obtained and the node fails with the above error.

In this situation (and with group=true in settings) it may be more appropriate to get a transaction-level advisory lock rather than a session-level advisory lock by using pg_try_advisory_xact_lock instead of pg_try_advisory_lock, which may be sufficient to get this working with PgBouncer transaction pooling.

[hacst]

Could someone with more knowledge on flyway internals elaborate on why session locks are required here? https://flywaydb.org/documentation/learnmore/faq#parallel mentions it for coordinating multiple nodes. However wouldn’t they coordinate themselves even with just per migration transaction level locking as each migration is done separately? Would something fundamental break if it was optionally allowed to work this way?

I also tried to check how other tools handle this and it seems like liquibase uses a more manual approach to locking using a table https://docs.liquibase.com/concepts/basic/databasechangeloglock-table.html . That should not be a problem with pgbouncer however it might leave itself locked accidentally if something goes wrong.

[fml2]

Another information that might be of interest:

• If PgBouncer is configured to work in transaction pooling mode, the error described in #3086 arises (can’t release advisory lock).

• If PgBouncer is configured to work in session pooling mode, everything works correctly, i.e. the advisory lock can be released at the end of the flyway migration.