logging methods no longer take a callback, can't reliably use it in some environments (AWS Lambda)

In Winston 3, the logging functions no longer take a callback. Previously this could be used to wait until logging had completed:

logger.info('some message', { foo: 42 }, callback);  // Winston 2.x

Winston 3 does allow you to listen to the logged event on your transports, but that doesn’t give me any easy way to tell when this message that I am currently logging, has completed. Since everything is async the next logged event that occurs after you write your message may not be related to the message you just logged. And it’s more complex if you are listening for the event on multiple transports.

That makes Winston difficult to use in certain environments. For example, in AWS Lambda we set the callbackWaitsForEmptyEventLoop parameter to false (documentation). (sidenote: you do this if you have a database connection or something that cannot be unrefed, because otherwise Lambda will never freeze your process). If set to false, Lambda freezes your process as soon as you return results to the caller and may even terminate your process. If your logging transport hasn’t finished writing by the time that happens, then you either lose logs or (worse) the logs get written later when Lambda unfreezes your process.

TL;DR: Our Lambda process would normally await (or the callback equivalent) on Winston before returning results to the caller, guaranteeing that logging is done before the process is frozen.

Is there another way you’d recommend to detect when logging is complete?

[dpraul]

I keep getting alerts on this thread, so figure I’ll share why I think this still isn’t working for a lot of people. Background: our use-case (and the context wherein this issue was opened) is AWS Lambda, so what I talk about here only applies there.

Lambda runs Node under the Lambda Execution Context. Pertinent information being:

After a Lambda function is executed, AWS Lambda maintains the execution context for some time in anticipation of another Lambda function invocation.

and

• Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again.

i.e. to speed up launches, Lambda “freezes” and “thaws” environments instead of fully shutting them down. To do this, Lambda doesn’t wait for Node to exit, it waits for your handler function to exit. Any asynchronous processes are run outside of the handler function, and thus are apt to be frozen with the rest of the Lambda Execution Context if they are not waited for.

Now let’s look at the suggested solution for waiting for Winston - adapted from UPGRADE-3.0.md, assuming we’re running in Lambda:

async function lambdaHandler(event, context) {
  const logger = winston.createLogger({
    transports: [
      new winston.transports.Console(),
      new CustomAsyncTransport(),
    ],
  });
  logger.log('info', 'some message');
  logger.on('finish', () => process.exit());
  logger.end();
}

Spot the problem? logger fires logger.end() within the handler function context, but the function fired by logger.on('finish') runs outside the handler context. Any async processes tied-up by CustomAsyncTransport will stall the finish event from firing, making it likely that the Execution Context freezes before that event fires.

To solve this, lambdaHandler must wait for the logger to exit before resolving:

async function lambdaHandler(event, context) {
  const logger = winston.createLogger({
    transports: [
      new winston.transports.Console(),
      new CustomAsyncTransport(),
    ],
  });
  const loggerFinished = new Promise(resolve => logger.on('finish', resolve));
  logger.log('info', 'some message');
  logger.end();
  await loggerFinished;
}

Since lambdaHandler doesn’t exit until logger fires the finish event, our CustomAsyncTransport should close before our lambda handler, saving those processes from being frozen (assuming the finish event is correctly implemented by @indexzero).

This can be generalized to something similar to the code I’ve shared previously:

async function waitForLogger(logger) {
  const loggerDone = new Promise(resolve => logger.on('finish', resolve));
  // alternatively, use end-of-stream https://www.npmjs.com/package/end-of-stream
  // although I haven't tested this
  // const loggerDone = new Promise(resolve => eos(logger, resolve));
  logger.end();
  return loggerDone;
}

async function lambdaHandler(event, context) {
  const logger = winston.createLogger({
    transports: [
      new winston.transports.Console(),
      new CustomAsyncTransport(),
    ],
  });
  logger.log('info', 'some message');
  await waitForLogger(logger);
}

Hope this helps some people.

[ScuroGuardiano]

@tsaockham I have solution 😃

function waitForLogger() {
    return new Promise(resolve => setTimeout(resolve, 2500));
}

After 2h of searching solution I implemented this, even function waitForLogger by @dpraul didn’t worked, I am done.