Cannot connect to AWS Aurora on latest version(s) due to EOFException

Describe the issue

Connection attempt to database fails for AWS Aurora Postgres due to EOFException on 42.2.16.

Downgrading to postgres driver version 42.2.14 works with no issues.

Driver Version?

42.2.16

Java Version?

11

OS Version?

Run in AWS Lambda

PostgreSQL Version?

PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit

To Reproduce

In my case, we have a Spring Boot server with Flyway and the program fails on startup (see logs below). So to reproduce you would start a Spring Boot (2.3.3) project with Flyway 6.5.5 and postgresql driver 42.2.16 and start it with a DB url pointing to an AWS Aurora cluster.

However, it seems that this is happening in other projects as well (not just Spring/Flyway). When googling for the error I found this issue in Quarkus: https://github.com/quarkusio/quarkus/issues/12022 , where several people describe the same error - without using Spring. One person even has the same issue against an Azure cluster. It seems to be postgres driver related, especially since downgrading to 42.2.14 solved the issue.

Expected behaviour

I expect the driver to connect to the database cluster without failing, as it does in previous versions.

Logs

These are the logs I can see in CloudWatch (unfortunately I cannot access the postgres logs)

2020-09-30 07:45:15  ERROR HikariPool:593 - HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: The connection attempt failed.
	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:315) ~[task/:?]
	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[task/:?]
	at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[task/:?]
	at org.postgresql.Driver.makeConnection(Driver.java:465) ~[task/:?]
	at org.postgresql.Driver.connect(Driver.java:264) ~[task/:?]
	at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[task/:?]
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358) ~[task/:?]
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) ~[task/:?]
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477) ~[task/:?]
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560) ~[task/:?]
	at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) ~[task/:?]
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[task/:?]
	at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56) ~[task/:?]
	at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:80) ~[task/:?]
	at org.flywaydb.core.Flyway.execute(Flyway.java:453) ~[task/:?]
	at org.flywaydb.core.Flyway.migrate(Flyway.java:158) ~[task/:?]
	at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:65) ~[task/:?]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1853) ~[task/:?]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1790) ~[task/:?]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[task/:?]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) ~[task/:?]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) ~[task/:?]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) [task/:?]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) [task/:?]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) [task/:?]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:311) [task/:?]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) [task/:?]
	at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1109) [task/:?]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869) [task/:?]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) [task/:?]
	at org.springframework.context.annotation.AnnotationConfigApplicationContext.<init>(AnnotationConfigApplicationContext.java:89) [task/:?]
	at com.alextherapeutics.lambda.AbstractStreamHandler.<init>(AbstractStreamHandler.java:55) [task/:?]
	at com.alextherapeutics.lambda.StreamLambdaHandler.<init>(StreamLambdaHandler.java:23) [task/:?]
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) [?:?]
	at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) [?:?]
	at java.lang.reflect.Constructor.newInstance(Unknown Source) [?:?]
	at lambdainternal.EventHandlerLoader.newInstance(EventHandlerLoader.java:409) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader$StreamMethodRequestHandler.fromMethod(EventHandlerLoader.java:333) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader$StreamMethodRequestHandler.makeRequestHandler(EventHandlerLoader.java:346) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader.getThreeLengthHandler(EventHandlerLoader.java:680) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader.getHandlerFromOverload(EventHandlerLoader.java:699) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader.loadEventPojoHandler(EventHandlerLoader.java:811) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.EventHandlerLoader.loadEventHandler(EventHandlerLoader.java:613) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.AWSLambda.findUserMethods(AWSLambda.java:122) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:226) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:192) [aws-lambda-java-runtime-0.2.0.jar:?]
	at lambdainternal.AWSLambda.main(AWSLambda.java:187) [aws-lambda-java-runtime-0.2.0.jar:?]
Caused by: java.io.EOFException
	at org.postgresql.core.PGStream.receiveChar(PGStream.java:443) ~[task/:?]
	at org.postgresql.core.v3.ConnectionFactoryImpl.enableGSSEncrypted(ConnectionFactoryImpl.java:436) ~[task/:?]
	at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:144) ~[task/:?]
	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213) ~[task/:?]
	... 47 more

[davecramer]

No worries, thanks for responding here with the root cause. It will help others!

[davecramer]

2 suggestions:

1. downgrade the driver to 42.2.14 or the like
2. very simple java program (without spring) that just connects and does select 1…