Recommendation on connection recycling

Based on the recommendation from AWS Aurora MySQL DBA Handbook

• Check and validate connections periodically even when they're not
borrowed. It helps detect and clean up broken or unhealthy connections
before an application thread attempts to use them.

• Don't let connections remain in the pool indefinitely. Recycle
connections by closing and reopening them periodically (for example,
every 15 minutes), which frees the resources associated with these
connections. It also helps prevent dangerous situations such as runaway
queries or zombie connections that clients have abandoned. This
recommendation applies to all connections, not just idle ones.

Can you please suggest a good way to recycle connections in a pool? Aurora has DNS based endpoints and any DNS changes do not reflect in the connections right away. Moreover, if there are multiple readers, creating a pool doesn’t uniformly distribute the connection load amongst the readers. If the connections are periodically reset (closed/reopened) before application accesses it, in the longer run, it can fully utilize multiple readers for read-scalability. I did not find a way where I can get all the connections from the pool and reset them. I can potentially loop through _allConnections.length, keep some state based on connectionId, but there’s no way to free a connection and not get it back again. If I recreate the pool, that seems very expensive to perform every 15 minutes and there could be other instances/functions using the pool to get connections at the time it’s recreated. I could potentially get a connection from the pool and remove it from the pool and close it. Would the pool recreate the connection lazily again in that case? It still won’t guarantee that every time this operation is run, the same connection is not closed and recreated.

Any suggestions are welcome, happy to create a PR if this could be a useful feature in longer run.

[nitsnwits]

Interesting, thanks for that input. I think recreating the pool could be the solution in that case. I’ll try that.

A follow-up suggestion (taken from the handbook link shared in the original description):

For the purposes of this whitepaper, a “smart driver” is a database driver or
connector with the ability to read DB cluster topology from the metadata table.
It can route new connections to individual instance endpoints without relying
on high-level cluster endpoints. A smart driver is also typically capable of load
balancing read-only connections across the available Aurora Replicas in a
round-robin fashion.

The MariaDB Connector/J is an example of a third-party Java Database
Connectivity (JDBC) smart driver with native support for Aurora MySQL DB
clusters. Application developers can draw inspiration from the MariaDB driver
to build drivers and connectors for languages other than Java.
See the MariaDB Connector/J page for details.2

Follow-up reference: https://mariadb.com/kb/en/failover-and-high-availability-with-mariadb-connector-j/#specifics-for-amazon-aurora

Full aurora support could be a different issue, but I’m suggesting that this could be a potential feature that every connection in the pool has a lifetime and is closed after that time automatically and a new connection is created lazily.

Sorry about the segue. Let me know if that’s interesting.

I’ll update this issue once I confirm recreating the pool helps in recycling connections.

[danielhinrichs86]

It would be great to see this or a similar add to the mysql2 pool to make it more cluster aware and “Aurora friendly”. When we horizontally scale our Aurora nodes the pool doesn’t pick the new nodes up in a timely fashion as idle connections don’t get regularly reestablished in order to query the Aurora endpoint to find the additional nodes.