Random ECONNRESET when talking to docker mysql

Hi all,

I have a big issue where I randomly get ECONNRESET every 1-5 minutes when using mysql / mysql2 npm package. ( Works fine up until this point )

Environment

• mysql server running in docker, mapped to host port 3306
• node server running within docker, talking to mysql via host_ip:3306

Errot

data-server-container |   Error: read ECONNRESET
data-server-container |
data-server-container |   - net.js:622 TCP.onread
data-server-container |     net.js:622:25

Fix attempts

I’ve tried all sorts to try and get the connection to remain - including

Using pools

const mysql = require('mysql2');
const connection = await mysql.createPool({
  connectionLimit: 10,
  host: process.env.DB_HOST_DOCKER,
  user: process.env.DB_USER_DOCKER,
  password: process.env.DB_PASSWORD_DOCKER,
  database: process.env.DB_DATABASE,
});

query = (sql) => {
  return new Promise(async (resolve, reject) => {
    connection.getConnection(function (err, poolConnection) {
      if (err) throw err; // not connected!

      // Use the connection
      poolConnection.query(sql, function (error, results, fields) {

        if (error) reject(error)

        resolve(results);

        // When done with the connection, release it.
        poolConnection.release();

      })

    })
  })
}

Creating a new connection on every request! ( Somehow this still gets ECONNRESET! )

query = (sql) => {

  const mysql = require('mysql2');

  const connection = await mysql.createConnection({
    host: process.env.DB_HOST_DOCKER,
    user: process.env.DB_USER_DOCKER,
    password: process.env.DB_PASSWORD_DOCKER,
    database: process.env.DB_DATABASE,
  });
  
  return new Promise(async (resolve, reject) => {
    connection.query(sql, (error, results) => {
      if (error) {
        reject(error);
      }
      resolve(results);
    });
  }

}

• Sending a ping to the connection every 10 seconds
• Checking my wait_timeout is high enough on mysql server ( 28800 as expected )

I’m running out of ideas here it’s stopping me being able to use this in production 😦

My best assumption after trying all the above is there an issue with the TCP Keep-Alive on the socket perhaps that is only triggered when running node / mysql within docker ( I stumbled on this MR - could this be a solution? https://github.com/mysqljs/mysql/pull/2110/ )

[hassan-jahan]

The problem is related to docker swarm and pooling.

Disable pooling (in knex file : pool: {min:0} )

Or check out this: https://success.docker.com/article/ipvs-connection-timeout-issue

[erming]

I had the same problem until I added command: "--wait_timeout=28800" to my docker-compose file. I’m running MariaDB though, but it might be the same problem.

Example docker-compose:

version: "3.7"
services:
  mysql:
    image: mariadb
    ports:
      - 3306:3306
    volumes:
      - mysql-volume:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: foo
      MYSQL_DATABASE: bar
    restart: always
    command: "--wait_timeout=28800"
volumes:
  mysql-volume:

Here’s a link to the MariaDB documentation:
https://mariadb.com/kb/en/server-system-variables/#wait_timeout