Stop providing pads after a long time

Hello all I really need your help. I’m hosting etherpad lite on my personal server and I got problems after some running time.

After some time, I can access the homepage but not pads anymore. I never have a response from the server. Traefik give me a “Bad Gateway” after some minutes. I don’t really have logs from supervisord. If I stop the docker stack and re-deploy it, pads are available again.

At the moment, I just re-started the stack so everything is good. I’ll update here when it will freeze again.

I join here all my docker swarm configuration. Maybe I missed something.

TY very much for the help. Kiss!

docker-compose.yml

---
version: '3.3'

services:

  app:
    image: etherpad/etherpad
    configs:
      - source: app-settings
        target: /opt/etherpad/settings.json
    volumes:
      - app-storage:/opt/etherpad
      - /etc/localtime:/etc/localtime:ro
    deploy:
      labels:
        traefik.http.frontend.rule: "Host:pad.misterbanal.net"
        traefik.http.frontend.entryPoints: "http"
        traefik.http.port: 9001
        traefik.http.frontend.redirect.entryPoint: "https"
        traefik.https.frontend.rule: "Host:pad.misterbanal.net"
        traefik.https.frontend.entryPoints: "https"
        traefik.https.port: 9001
        traefik.docker.network: web-proxy
    networks:
      db: ~
      web-proxy: ~

  db:
    image: postgres:11-alpine
    volumes:
      - db-storage:/var/lib/postgresql/data
      - /etc/localtime:/etc/localtime:ro
    environment:
      POSTGRES_USER: etherpad
      POSTGRES_PASSWORD: etherpad
      POSTGRES_DB: etherpad
    networks:
      db: ~

volumes:
  app-storage: ~
  db-storage: ~

configs:
  app-settings:
    file: ./docker/config/settings.json

networks:
  db: ~
  web-proxy:
    external: true

settings.yml

/*
 * This file must be valid JSON. But comments are allowed
 *
 * Please edit settings.json, not settings.json.template
 *
 * Please note that starting from Etherpad 1.6.0 you can store DB credentials in
 * a separate file (credentials.json).
 *
 *
 * ENVIRONMENT VARIABLE SUBSTITUTION
 * =================================
 *
 * All the configuration values can be read from environment variables using the
 * syntax "${ENV_VAR_NAME}".
 * This is useful, for example, when running in a Docker container.
 *
 * EXAMPLE:
 *    "port":     "${PORT}"
 *    "minify":   "${MINIFY}"
 *    "skinName": "${SKIN_NAME}"
 *
 * Would read the configuration values for those items from the environment
 * variables PORT, MINIFY and SKIN_NAME.
 *
 * REMARKS:
 * Please note that a variable substitution always needs to be quoted.
 *    "port":   9001,          <-- Literal values. When not using substitution,
 *    "minify": false              only strings must be quoted: booleans and
 *    "skin":   "colibris"         numbers must not.
 *
 *    "port":   ${PORT}        <-- ERROR: this is not valid json
 *    "minify": ${MINIFY}
 *    "skin":   ${SKIN_NAME}
 *
 *    "port":   "${PORT}"      <-- CORRECT: if you want to use a variable
 *    "minify": "${MINIFY}"        substitution, put quotes around its name,
 *    "skin":   "${SKIN_NAME}"     even if the required value is a number or a
 *                                 boolean.
 *                                 Etherpad will take care of rewriting it to
 *                                 the proper type if necessary.
 */
{
  /*
   * Name your instance!
   */
  "title": "The Banal Pad",

  /*
   * favicon default name
   * alternatively, set up a fully specified Url to your own favicon
   */
  "favicon": "favicon.ico",

  /*
   * Skin name.
   *
   * Its value has to be an existing directory under src/static/skins.
   * You can write your own, or use one of the included ones:
   *
   * - "no-skin":  an empty skin (default). This yields the unmodified,
   *               traditional Etherpad theme.
   * - "colibris": the new experimental skin (since Etherpad 1.8), candidate to
   *               become the default in Etherpad 2.0
   */
  "skinName": "no-skin",

  /*
   * IP and port which etherpad should bind at
   */
  "ip": "0.0.0.0",
  "port" : 9001,

  /*
   * Option to hide/show the settings.json in admin page.
   *
   * Default option is set to true
   */
  "showSettingsInAdminPage" : true,

  /*
   * Node native SSL support
   *
   * This is disabled by default.
   * Make sure to have the minimum and correct file access permissions set so
   * that the Etherpad server can access them
   */

  /*
  "ssl" : {
            "key"  : "/path-to-your/epl-server.key",
            "cert" : "/path-to-your/epl-server.crt",
            "ca": ["/path-to-your/epl-intermediate-cert1.crt", "/path-to-your/epl-intermediate-cert2.crt"]
          },
  */

  /*
   * The type of the database.
   *
   * You can choose between many DB drivers, for example: dirty, postgres,
   * sqlite, mysql.
   *
   * You shouldn't use "dirty" for for anything else than testing or
   * development.
   *
   *
   * Database specific settings are dependent on dbType, and go in dbSettings.
   * Remember that since Etherpad 1.6.0 you can also store these informations in
   * credentials.json.
   *
   * For a complete list of the supported drivers, please refer to:
   * https://www.npmjs.com/package/ueberdb2
   */

   /*
  "dbType" : "dirty",
  "dbSettings" : {
                   "filename" : "var/dirty.db"
                 },
   */

  /*
   * An Example of MySQL Configuration (commented out).
   *
   * See: https://github.com/ether/etherpad-lite/wiki/How-to-use-Etherpad-Lite-with-MySQL
   */

  "dbType" : "postgres",
  "dbSettings" : {
      "user"    : "etherpad",
      "host"    : "db",
      "port"    : 5432,
      "password": "etherpad",
      "database": "etherpad",
      "charset" : "utf8mb4"
  },

  /*
   * The default text of a pad
   */
  "defaultPadText" : "Welcome to Etherpad!\n\nThis pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!\n\nGet involved with Etherpad at http:\/\/etherpad.org\n",

  /*
   * Default Pad behavior.
   *
   * Change them if you want to override.
   */
  "padOptions": {
    "noColors": false,
    "showControls": true,
    "showChat": true,
    "showLineNumbers": true,
    "useMonospaceFont": false,
    "userName": false,
    "userColor": false,
    "rtl": false,
    "alwaysShowChat": false,
    "chatAndUsers": false,
    "lang": "en-gb"
  },

  /*
   * Pad Shortcut Keys
   */
  "padShortcutEnabled" : {
    "altF9"     : true, /* focus on the File Menu and/or editbar */
    "altC"      : true, /* focus on the Chat window */
    "cmdShift2" : true, /* shows a gritter popup showing a line author */
    "delete"    : true,
    "return"    : true,
    "esc"       : true, /* in mozilla versions 14-19 avoid reconnecting pad */
    "cmdS"      : true, /* save a revision */
    "tab"       : true, /* indent */
    "cmdZ"      : true, /* undo/redo */
    "cmdY"      : true, /* redo */
    "cmdI"      : true, /* italic */
    "cmdB"      : true, /* bold */
    "cmdU"      : true, /* underline */
    "cmd5"      : true, /* strike through */
    "cmdShiftL" : true, /* unordered list */
    "cmdShiftN" : true, /* ordered list */
    "cmdShift1" : true, /* ordered list */
    "cmdShiftC" : true, /* clear authorship */
    "cmdH"      : true, /* backspace */
    "ctrlHome"  : true, /* scroll to top of pad */
    "pageUp"    : true,
    "pageDown"  : true
  },

  /*
   * Should we suppress errors from being visible in the default Pad Text?
   */
  "suppressErrorsInPadText" : false,

  /*
   * If this option is enabled, a user must have a session to access pads.
   * This effectively allows only group pads to be accessed.
   */
  "requireSession" : false,

  /*
   * Users may edit pads but not create new ones.
   *
   * Pad creation is only via the API.
   * This applies both to group pads and regular pads.
   */
  "editOnly" : false,

  /*
   * If set to true, those users who have a valid session will automatically be
   * granted access to password protected pads.
   */
  "sessionNoPassword" : false,

  /*
   * If true, all css & js will be minified before sending to the client.
   *
   * This will improve the loading performance massively, but makes it difficult
   * to debug the javascript/css
   */
  "minify" : true,

  /*
   * How long may clients use served javascript code (in seconds)?
   *
   * Not setting this may cause problems during deployment.
   * Set to 0 to disable caching.
   */
  "maxAge" : 21600, // 60 * 60 * 6 = 6 hours

  /*
   * Absolute path to the Abiword executable.
   *
   * Abiword is needed to get advanced import/export features of pads. Setting
   * it to null disables Abiword and will only allow plain text and HTML
   * import/exports.
   */
  "abiword" : null,

  /*
   * This is the absolute path to the soffice executable.
   *
   * LibreOffice can be used in lieu of Abiword to export pads.
   * Setting it to null disables LibreOffice exporting.
   */
  "soffice" : null,

  /*
   * Path to the Tidy executable.
   *
   * Tidy is used to improve the quality of exported pads.
   * Setting it to null disables Tidy.
   */
  "tidyHtml" : null,

  /*
   * Allow import of file types other than the supported ones:
   * txt, doc, docx, rtf, odt, html & htm
   */
  "allowUnknownFileEnds" : true,

  /*
   * This setting is used if you require authentication of all users.
   *
   * Note: "/admin" always requires authentication.
   */
  "requireAuthentication" : false,

  /*
   * Require authorization by a module, or a user with is_admin set, see below.
   */
  "requireAuthorization" : false,

  /*
   * When you use NGINX or another proxy/load-balancer set this to true.
   */
  "trustProxy" : true,

  /*
   * Privacy: disable IP logging
   */
  "disableIPlogging" : true,

  /*
   * Time (in seconds) to automatically reconnect pad when a "Force reconnect"
   * message is shown to user.
   *
   * Set to 0 to disable automatic reconnection.
   */
  "automaticReconnectionTimeout" : 0,

  /*
   * By default, when caret is moved out of viewport, it scrolls the minimum
   * height needed to make this line visible.
   */
  "scrollWhenFocusLineIsOutOfViewport": {

    /*
     * Percentage of viewport height to be additionally scrolled.
     *
     * E.g.: use "percentage.editionAboveViewport": 0.5, to place caret line in
     *       the middle of viewport, when user edits a line above of the
     *       viewport
     *
     * Set to 0 to disable extra scrolling
     */
    "percentage": {
      "editionAboveViewport": 0,
      "editionBelowViewport": 0
    },

    /*
     * Time (in milliseconds) used to animate the scroll transition.
     * Set to 0 to disable animation
     */
    "duration": 0,

    /*
     * Flag to control if it should scroll when user places the caret in the
     * last line of the viewport
     */
    "scrollWhenCaretIsInTheLastLineOfViewport": false,

    /*
     * Percentage of viewport height to be additionally scrolled when user
     * presses arrow up in the line of the top of the viewport.
     *
     * Set to 0 to let the scroll to be handled as default by Etherpad
     */
    "percentageToScrollWhenUserPressesArrowUp": 0
  },

  /*
   * Users for basic authentication.
   *
   * is_admin = true gives access to /admin.
   * If you do not uncomment this, /admin will not be available!
   *
   * WARNING: passwords should not be stored in plaintext in this file.
   *          If you want to mitigate this, please install ep_hash_auth and
   *          follow the section "secure your installation" in README.md
   */

  "users": {
    "admin": {
      // "password" can be replaced with "hash" if you install ep_hash_auth
      "password": "XXXXXXXXXXXXXXXXXXXXXXXXX",
      "is_admin": true
    },
  },

  /*
   * Restrict socket.io transport methods
   */
  "socketTransportProtocols" : ["xhr-polling", "jsonp-polling", "htmlfile"],

  /*
   * Allow Load Testing tools to hit the Etherpad Instance.
   *
   * WARNING: this will disable security on the instance.
   */
  "loadTest": false,

  /*
   * Disable indentation on new line when previous line ends with some special
   * chars (':', '[', '(', '{')
   */

  /*
  "indentationOnNewLine": false,
  */

  /*
   * Toolbar buttons configuration.
   *
   * Uncomment to customize.
   */

  /*
  "toolbar": {
    "left": [
      ["bold", "italic", "underline", "strikethrough"],
      ["orderedlist", "unorderedlist", "indent", "outdent"],
      ["undo", "redo"],
      ["clearauthorship"]
    ],
    "right": [
      ["importexport", "timeslider", "savedrevision"],
      ["settings", "embed"],
      ["showusers"]
    ],
    "timeslider": [
      ["timeslider_export", "timeslider_returnToPad"]
    ]
  },
  */

  /*
   * The log level we are using.
   *
   * Valid values: DEBUG, INFO, WARN, ERROR
   */
  "loglevel": "ERROR",

  /*
   * Logging configuration. See log4js documentation for further information:
   * https://github.com/nomiddlename/log4js-node
   *
   * You can add as many appenders as you want here.
   */
  "logconfig" :
    { "appenders": [
        { "type": "console"
        //, "category": "access"// only logs pad access
        }

      /*
      , { "type": "file"
      , "filename": "your-log-file-here.log"
      , "maxLogSize": 1024
      , "backups": 3 // how many log files there're gonna be at max
      //, "category": "test" // only log a specific category
        }
      */

      /*
      , { "type": "logLevelFilter"
        , "level": "warn" // filters out all log messages that have a lower level than "error"
        , "appender":
          {  Use whatever appender you want here  }
        }
      */

      /*
      , { "type": "logLevelFilter"
        , "level": "error" // filters out all log messages that have a lower level than "error"
        , "appender":
          { "type": "smtp"
          , "subject": "An error occurred in your EPL instance!"
          , "recipients": "[e-mail], [e-mail]"
          , "sendInterval": 300 // 60 * 5 = 5 minutes -- will buffer log messages; set to 0 to send a mail for every message
          , "transport": "SMTP", "SMTP": { // see https://github.com/andris9/Nodemailer#possible-transport-methods
              "host": "smtp.example.com", "port": 465,
              "secureConnection": true,
              "auth": {
                  "user": "[e-mail]",
                  "pass": "bar_foo"
              }
            }
          }
        }
      */

      ]
    } // logconfig
}

[muxator]

Hi @eluminae, thanks for the report.

I guess we are facing a limitation of the current docker image, where supervisor does not capture the stdout of etherpad, and the logs with the real error get lost.

Moving forward, I think we are going to get rid of supervisor entirely. In that case, you would see the container stopping (which I personally like: supervision should be done with an external mechanism anyways) and inspecting the reason for the crash would be straightforward, since docker logs would show etherpad’s logs.

Back to your problem: there is certainly something happening in ethrrpad itself, and it would be good to understand why.

I hope in the following days (probably next weekend) to be able to come back to you with a way to further diagnose the problem.

[JohnMcLear]

Closing as stale