Winston 3.0.0 still halts logging in high stress. Makes it unusable for us.

Please tell us about your environment:

• version winston@3.0.0
• node -v outputs: v8.10.0
• Operating System: Ubuntu 18.04
• Language: ES6

This issue has been closed, but running the following code (which I also posted here) still stops Winston logs from working within a second or so:

"use strict";

let winston = require('winston');

const myFormat = winston.format.printf(info => {
    return `${info.timestamp} ${info.level}: ${info.message}`;
});
const logger = winston.createLogger({
    level: 'info',
    format: winston.format.timestamp(),
    transports: [
        new winston.transports.File({filename: 'logs/combined.log'}),
        new winston.transports.Console({format: winston.format.combine(winston.format.timestamp(), myFormat), level: 'debug'})
    ]
});

while (true) {
    let random_string = Math.random().toString(36).substring(7);
    logger.info(random_string);  // This one stops after a while
    console.log(random_string);  // while this one continues to function properly
}

[kollerebbe]

@mempf We had the same problem within our project. We use “bottleneck” to prevent the problem. In our case, we sometimes log a bunch of entries at once and winston hung up in the past. We use bottleneck to “queue” calls of the log function. This fixed our problem. Our LoggerService looks like this (typescript):

import * as winston from 'winston';
import * as bottleneck from 'bottleneck';

const { printf, combine, colorize } = winston.format;

const logLevels = {
  levels: {
    error: 0,
    warn: 1,
    info: 2,
    http: 3,
    sql: 4,
    debug: 5
  },
  colors: {
    error: 'red',
    warn: 'yellow',
    info: 'green',
    http: 'blue',
    sql: 'blue',
    debug: 'gray'
  }
};
winston.addColors(logLevels);

export const jsonFormatter = (logEntry: any) => {
  const base = { timestamp: new Date() };
  const json = Object.assign(base, logEntry);
  const today = new Date();
  const day = today.toISOString().split('T')[0];
  const hours = today.getUTCHours();
  const minutes = today.getUTCMinutes();
  logEntry['message'] = `[${day} ${hours}:${minutes} UTC]: ${json.message}`;
  return logEntry;
};

/**
 * Logger Service. Handles logging with winston. Service handles logging time intervals with bottleneck.
 * Bottleneck, because winston will hang up under heavy load.
 */
export const LoggerService = {
  /** "Bottleneck" Limiter for Logger */
  bottleneck: new bottleneck.default({
    maxConcurrent: 1,
    minTime: 5
  }),
  /** Winston logger. See https://github.com/winstonjs/winston#creating-your-own-logger for more details  */
  logger: winston.createLogger({
    level: 'info',
    format: combine(
      colorize(),
      winston.format(jsonFormatter)(),
      printf((info: any) => `${info.level}: ${info.message}`)
    ),
    transports: [
      new winston.transports.File({ filename: 'error.log', level: 'error' }),
      new winston.transports.File({ filename: 'combined.log' })
    ],
    exceptionHandlers: [
      new winston.transports.File({ filename: 'exceptions.log' })
    ]
  }),
  /** Returns winston logger */
  getLogger() {
    return LoggerService.logger;
  },
  /**
   * Logs an event with winston
   * @param param0.level Log level. e.g. warn, info, error
   * @param param0.message Log message
   */
  log({ level, message } : {level: string, message: string }) {
    LoggerService.bottleneck.schedule({}, () => {
      return Promise.resolve(LoggerService.getLogger().log({ level, message }));
    });
  }
};

LoggerService.logger.add(new winston.transports.Console({
  format: winston.format.simple()
}));

[yinzara]

I believe I have found a work around and it’s a lot simpler then you might think. It turns out that “readable-stream” automatically "unpipe"s the writable stream on any transport error.

Workaround:

const winston = require("winston")
const transport = newErrorProneTransport() // FileTransport is only one, there are many
const logger = winston.createLogger({
    level: "info",
    transports: [transport]
})
logger.on("error", error => {
    //if your JS version doesn't have "includes(transport)", use another method like `.some(t => t===transport)`
    if (!logger.transports.includes(transport)) {
        logger.add(transport)
    }
})

Here is the test now correctly passing

const winston = require("winston")
describe("transports issue", () => {
    const mainError = "Error logging!"
    const otherError = "Other error"
    let logger
    let errorMessage
    let counter
    let maxCounter
    let logError
    let transport
    const newTransport = () =>
        Object.assign(new winston.transports.Console(), {
            log: (info, next) => {
                if (counter === maxCounter) {
                    next(new Error(errorMessage))
                } else {
                    if (logError !== undefined) {
                        errorMessage = otherError
                    }
                    counter = counter + 1
                    next()
                }
            }
        })
    beforeEach(() => {
        errorMessage = mainError
        counter = 0
        maxCounter = 1
        logError = undefined
        transport = newTransport()
        logger = winston.createLogger({
            level: "info",
            transports: [transport]
        })
        logger.on("error", error => {
            if (!logger.transports.includes(transport)) {
                logger.add(transport)
                counter = 0
            }
            logError = error
        })
    })

    describe("only log once", () => {
        beforeEach(() => {
            logger.info("log once")
        })

        test("pipes is transport", () => {
            expect(logger.transports).toEqual([transport])
        })

        test("error didn't", () => {
            expect(logError).toBeUndefined()
        })
    })

    describe("log twice", () => {
        beforeEach(() => {
            logger.info("log twice 1")
            logger.info("log twice 2") // this raises the `mainError` for the transport
        })

        test("pipes is transport", () => {
            expect(logger.transports).toEqual([transport])
        })

        test("error occurred", () => {
            expect(logError).toHaveProperty("message", mainError)
        })
    })

    describe("log thrice", () => {
        beforeEach(() => {
            logger.info("log thrice 1")
            logger.info("log thrice 2") // this raises the `mainError` for the transport
            logger.info("log thrice 3") 
        })

        test("pipes is transport", () => {
            expect(logger.transports).toEqual([transport])
        })

        test("error occurred", () => {
            expect(logError).toHaveProperty("message", mainError)
        })
    })

    describe("log four times", () => {
        beforeEach(() => {
            logger.info("log four times 1")
            logger.info("log four times 2") // this raises the `mainError` for the transport
            logger.info("log four times 3")
            logger.info("log four times 4") // this raises the `otherError` for the transport
        })

        test("pipes is transport", () => {
            expect(logger.transports).toEqual([transport])
        })

        test("other error occurred", () => {
            expect(logError).toHaveProperty("message", otherError)
        })
    })
})