max_prepared_stmt_count error, do I need to increase maxPreparedStatements or is something wrong with it?

I’m getting the below error from mysql:

Can\'t create more than max_prepared_stmt_count statements (current value: 16382)

I’ve increased the value to the max (1 million) for now.

I’m thinking even though it has increased for now, no matter what my prepared statement count will eventually hit 1M.

What I don’t understand is that I see a default value for maxPreparedStatements for 16000. Wouldn’t that prevent this issue from happening? Is it because I have 3 replica servers connected to my mysql instance? If so, if I increase my max_prepared_stmt_count statements to something > 16000 * 3, like for example 50000, will that prevent this problem from reoccuring?

Also how do I clear this cache in the meantime if I hit this 1M mark?

My node/mysql2 code is as follows:

import mysql from 'mysql2/promise'
const pool = mysql.createPool({ host, port, connectionLimit: 100 })
export default pool

I then use the pool everywhere with prepared statements a la:

pool.execute('SELECT * FROM ... etc etc')

[terencechow]

Ok after some testing I have figured a possible solution. Some preliminary analysis here, but I’ll leave this for future users.

maxPreparedStatements is not per server, it is per connection. max_prepared_stmt_count is across all connections.

Hence if I have a max_prepared_stmt_count of 20 and I have 1 server with a connectionLimit of 4, then having maxPreparedStatements of 5 fails. However having a maxPreparedStatements of 4 does not fail. This is because the maxPreparedStatements become 20 and 16 respectively. Somehow 20 causes an error on the next prepare statement but 16 causes the LRU to refresh without any errors. Might be a minor bug somewhere where the value is checking <= instead of just < ? (It might be that I make multiple concurrent requests and they concurrently prepare statements as well, hence a race condition causes an error…)

This is testable by creating a route that prepares a bunch of statements.

With maxPreparedStatements of 4 and connectionLimit of 4, show global status like '%Prepared_stmt_count%'; gives a maximum of 16 after a number of requests to that route

With maxPreparedStatements of 5 and connectionLimit of 4, show global status like '%Prepared_stmt_count%'; gives a maximum of 20, but the next call to prepare a query gives me an error.

[danielecr]

I do not have a reproducible test case, but I had this problem on one service and I solved it by lowering the maxPreparedStatements to 20, and connection pool to 1. My requirement was very specific, maybe, because trying to make parallel query execution of the same query cause a big lock on the table, so it make no sense to have more than one connection running per time, but having pool warrant me against connection lost for long idle time. I also have php code where I use prepared statement (real, not emulated), and I used a stupid solution for the reuse of prepared statement, indexing it by:

$prepared[` md5($sqlStatement) ] = mysql_prepare($sqlStatement);

and reusing the prepared staff every time.

I expected something similar with pool connection, like connId + "." + md5(sql) so I have a look at the code, I find it is used LRU cache (that is very cool staff with dispose, nice to know), and that every connection has its own lru cache (so connId was a bad expectation), but the key is so complex, I expected to read unique association between the sql prepare statement (I mean https://dev.mysql.com/doc/refman/8.0/en/prepare.html ) and the string, but I find there is an options where sql (string?) is just one part and nestTable is preparsed and used.

I do not know what exactly is needed for and why parsing is needed, but I know my code generate and use at most 5 prepared statements, and it is not possible to exeeds the max_prepared_stmt_count with 5 prepared and 20 connections. (If I recall correctly, I changed that code because it breaks everything)

What happens if I replace statementKey(options) code with just return options.sql?