question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Asynchronous file logging writes out-of-order entries

See original GitHub issue

The implementation of asynchronous (sync = false) file logging doesn’t guarantee the logs are written out in order.

The (abbreviated) code looks like this:

writeLine(text) {
   ...
  this.asyncWriteQueue.push(text);
  this.nextAsyncWrite();
}

nextAsyncWrite() {
    var text = this.asyncWriteQueue.shift();
   fs.writeFile(..., err => {
         this.nextAsyncWrite();
   });
}

When writeLine() is called multiple times, it results in multiple in-flight fs.writeFile() calls - the order in which these calls complete is not guaranteed (that is, the second fs.writeFile() might complete before the first one, depending on I/O scheduling).

This is demonstrated by the following:

const logger = require('electron-log');
logger.transports.file.sync = false;

for (let i=0; i < 100; i++) {
  logger.info(`${i}`);
}

It should log the numbers in order in the file, but doesn’t.

(Also note that the code is quite inefficient - it calls fs.writeFile() for every single log line pushed to asyncWriteQueue - it should probably just write all the buffered entries in a single write each time nextAsyncWrite() is called.)

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
megahertzcommented, Oct 6, 2021

@ramaralo This task has low priority for me, so it won’t be changed quickly

1reaction
xmedekocommented, Apr 12, 2021

Note: pino hooks handlers to beforeExit, exit, uncaughtException, SIGHUP, SIGINT, SIGQUIT, SIGTERM to flush async logger, see https://github.com/pinojs/pino/blob/master/docs/asynchronous.md#caveats (But I do not find it necessary.)

Read more comments on GitHub >

github_iconTop Results From Across the Web

When to write asynchronous logs - c++ - Stack Overflow
Problem here is that sorting your log entries into a vector might introduce a lot of overhead into your logger if you don't...
Read more >
[#LOG4J2-2031] Messages appear out of order in log file (was
The log4j2.xml uses an asynchronous appender. What I observe in the output log file is that after a short interval (120 or so...
Read more >
How Do I maintain the chronological order in which Asynchronous ...
1. The first method simply obtains a "place holder" (the long Order variable) then asynchronously calls a method to write the log file...
Read more >
Asynchronous logging : r/rust - Reddit
That being said, I'm writing a file appender in tracing because we'd like ... problems with logs out of order with a naive...
Read more >
Configure Logging Mode (Synchronous or Asynchronous)
In asynchronous mode, the logger writes entries to a queue, ... In contrast, synchronous mode might be slower for a logger writing to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found