question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

High concurrency writes leads to data loss

See original GitHub issue

testing script

$ cat winston.js 
var winston = require('winston');

var logger = new (winston.Logger)({
        transports: [
          new (winston.transports.Console)(),
          new (winston.transports.File)({ filename: 'win.log' })
        ]
      });

var http=require('http');

var server = http.createServer(function(request, response){
    response.writeHead(200, {'Content-Type': 'text/plain'});
    var rd = Math.random() * 50;
    logger.info("hello " + rd);
    response.write('hello ');
    if (Math.floor(rd) == 30){
        process.exit(1);
    }
    response.end();
}).listen(4444);

ab testing command

ab -c 10 -n 100 localhost:4444/ 

console log result all request

$ node winston.js 
info: hello 40.635715366806835
info: hello 47.55378740373999
info: hello 3.97098318208009
info: hello 21.402712271083146
info: hello 13.45938507001847
info: hello 34.73868641303852
info: hello 35.19469986204058
info: hello 17.50683527207002
info: hello 46.6102683218196
info: hello 16.208414710126817
info: hello 18.39921014616266
info: hello 23.423498892225325
info: hello 44.21681684907526
info: hello 35.54235898191109
info: hello 9.904313215520233
info: hello 27.047289244364947
info: hello 32.58601982379332
info: hello 36.48257440654561
info: hello 42.66765915090218
info: hello 9.47293157223612
info: hello 38.31890656147152
info: hello 40.63100240891799
info: hello 26.544194167945534
info: hello 17.846915975678712
info: hello 43.33941290387884
info: hello 26.012209930922836
info: hello 30.679881805554032

But file log only has 3 result:

$ cat win.log 
{"level":"info","message":"hello 40.635715366806835","timestamp":"2013-07-30T03:03:43.441Z"}
{"level":"info","message":"hello 47.55378740373999","timestamp":"2013-07-30T03:03:43.446Z"}
{"level":"info","message":"hello 3.97098318208009","timestamp":"2013-07-30T03:03:43.446Z"}

Issue Analytics

  • State:closed
  • Created 10 years ago
  • Reactions:1
  • Comments:14 (4 by maintainers)

github_iconTop GitHub Comments

3reactions
destructivecreatorcommented, Jan 6, 2018

“winston”: “^3.0.0-rc1” facing the same issue. Logs go away under high concurrency writes. Any comments??

0reactions
indexzerocommented, Jun 1, 2018

Very. We have appropriate test coverage ensuring this. See: https://github.com/winstonjs/winston/blob/master/test/transports/00-file-stress.test.js.

Locking issue due to age.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Database Concurrency Conflicts in the Real World
Because if user 1 writes his data back into the database he will overwrite the changes made by user 2, causing them to...
Read more >
Concurrency problems - theory and experimentation in SQL ...
This leads to a loss for the update statement performed by second transaction. As Transaction 1 overwrites a value that Transaction 2 ...
Read more >
A Deep Dive into Database Concurrency Control
This article explores database concurrency control and discusses various concurrency control methods such as 2PL, T/O, OCC, and MVCC.
Read more >
Ensuring data(base) consistency during concurrent requests
This post will focus on inconsistencies caused by concurrent requests that result in an incorrect or unexpected state in the database. Guarding ...
Read more >
Data loss on concurrent file write in camel - Stack Overflow
And the implementation can be different depending on OS platform as well. This could lead to that Camel thinks the file is not...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found