question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

AWS RDS and Extended Query Protocol

See original GitHub issue

Hi! Amazing library, thanks for all of the hard work.

TLDR: The extended query protocol makes the AWS RDS Proxy entirely useless. Is there any way to turn off the extended query protocol in pg?

I have a serious issue I’m facing in a serverless environment. I’m using AWS Lambda, and directly connecting to postgres has caused horrific performance and stability issues because of the vast numbers of connections being opened and closed. So, the solution from AWS seems to be to use their RDS Proxy. It sits in front of postgres and handles a warm connection pool. I’ve set it all up properly. Unfortunately, all of my connections are being session pinned, essentially rendering the proxy useless, and it even seems to be performing worse than without the proxy. I’ve looked at my error logs and found this error repeatedly:

The client session was pinned to the database connection [dbConnection=251482716] for the remainder of the session. The proxy can't reuse this connection until the session ends. Reason: A parse message was detected.

Reason: A parse message was detected.. This seems to come because of the extended query protocol. Apparently the extended query protocol causes pinning, as documented here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy.html#rds-proxy-pinning

So my real question is, is there any way to turn off the extended query protocol in pg? If not, does anyone have any insight into how I can properly manage connections to my database in a serverless environment? This is a major issue for us at this time, and I’m not sure how to proceed

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:18 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
brianccommented, Oct 4, 2020

But in node-postgres and I imagine other clients, parameterized queries are generally implemented as prepared statements.

There is no difference at the protocol level, actually. Any query with parameters (whether or not you call it a prepared statement) requires a parse/bind/describe/execute series of messages. Either way it “prepares” the query during the parse phase. Sorry the RDS proxy is so picky about parameterized queries…that’s quite limiting as doing parameters in queries with string concatenation and client side sanitization is not ideal. Let me know if there’s anything you can think of from this libraries side to help…but sounds like an RDS issue for now.

1reaction
lastmjscommented, Feb 4, 2022

@picosam I don’t really remember, I don’t work for that employer now and I’ve transitioned entirely into Web3. I think we created it outside of the handler, and I think we used a single connection, but really I don’t remember. I’m sure I did the most sensible way after trying out or researching many ways of doing it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Managing an RDS Proxy - Amazon Relational Database Service
Using the PostgreSQL extended query protocol such as by using JDBC default settings. Creating temporary sequences, tables, or views. Declaring cursors.
Read more >
Best practices for Amazon RDS - AWS Documentation
Amazon RDS basic operational guidelines; DB instance RAM recommendations ... When Enhanced Monitoring is enabled, Amazon RDS provides metrics in real time ...
Read more >
Amazon RDS for PostgreSQL - AWS Documentation
Cross-Region snapshot copy. Cross-Region read replicas. Creating a new DB instance in the preview environment. Use the following procedure to create a ...
Read more >
Using Amazon RDS Proxy - AWS Documentation
RDS Proxy doesn't multiplex connections when your client application drivers use the PostgreSQL extended query protocol. RDS Proxy currently doesn't support ...
Read more >
Using extended events with Amazon RDS for Microsoft SQL ...
Extended event file targets can only write files to the D:\rdsdbdata\log directory on RDS for SQL Server. As an example, use the following...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found