DatabaseError: (psycopg2.DatabaseError) server closed the connection unexpectedly

I’m getting errors occasionally causing users to have to reload the page a few times.

Looking the logs I have:

  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2820, in _execute_and_instances
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2797, in __iter__
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
    return self._execute_and_instances(context)
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2820, in _execute_and_instances
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    result = conn.execute(querycontext.statement, self._params)
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    return connection._execute_clauseelement(self, multiparams, params)
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    exc_info
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    context)
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 469, in do_execute
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    cursor.execute(statement, parameters)
DatabaseError: (psycopg2.DatabaseError) server closed the connection unexpectedly
    context)
     This probably means the server terminated abnormally
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 469, in do_execute
    cursor.execute(statement, parameters)
     before or while processing the request.
DatabaseError: (psycopg2.DatabaseError) server closed the connection unexpectedly

Redash version:

redash@40c099166fa0:/app$ head CHANGELOG.md
# Change Log

## v1.0.3 - 2017-04-18

### Fixed

- Fix: sort by column no longer working.

## v1.0.2 - 2017-04-18

I’m running Redash on docker swarm using two service configuration files:

dbs.yml:

version: '3.1'

services:
  postgres:
    image: mdillon/postgis:9.6
    volumes:
      - pgdata:/var/lib/postgresql/data
    networks:
      - pg
    environment:
      - POSTGRES_PASSWORD=$PG_DB_PASS
      - POSTGRES_USER=$PG_DB_USER
    ports:
      - '5432:5432'

  redis:
    image: redis:3.0.7
    volumes:
      - redisdata:/data
    networks:
      - redis

volumes:
  pgdata:
  redisdata:

networks:
  pg:
  redis:

redash.yml:

version: '3.1'

services:
  redash:
    image: redash/redash:latest
    command: server
    depends_on:
      - postgres
      - redis
    ports:
      - "5000:5000"
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://${PG_DB_USER}:${PG_DB_PASS}@postgres/redash"
      REDASH_COOKIE_SECRET: ${SECRET_KEY}
      REDASH_GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
      REDASH_GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET}
      REDASH_MAIL_SERVER: ${EMAIL_HOST}
      REDASH_MAIL_PORT: 587
      REDASH_MAIL_USE_TLS: "true"
      REDASH_MAIL_USE_SSL: "true"
      REDASH_MAIL_USERNAME: ${EMAIL_HOST_USER}
      REDASH_MAIL_PASSWORD: ${EMAIL_HOST_PASSWORD}
      REDASH_MAIL_DEFAULT_SENDER: ${EMAIL_HOST_USER}
    networks:
      pg:
      redis:

  redash-worker:
    image: redash/redash:latest
    command: scheduler
    depends_on:
      - postgres
      - redis
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://${PG_DB_USER}:${PG_DB_PASS}@postgres/redash"
      QUEUES: "queries,scheduled_queries,celery"
      WORKERS_COUNT: 2
    networks:
      pg:
      redis:

networks:
  pg:
  redis:

[RichardLitt]

This might be a good topic for the forum, @shankardevy and @rennopaat, as it isn’t clearly a redash issue but a setup (aka, support) issue. I would open a thread there and link it back here, even, so that we can discuss over there.

[arikfr]

I managed to reproduce this when I restarted the Postgres container. Redash maintains a connection pool and doesn’t verify that the connection is “alive” before using it. This shouldn’t happen often, but if your containers scheduler restarts the database container often you will encounter this more often.