question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error on streaming huge recordset

See original GitHub issue

Hi @patriksimek please consider following issue.

"mssql": "^3.3.0",

  mssql: Object.assign({}, {
        stream: true,
        options: {encrypt: false},
        debug: true,
        requestTimeout: 50000,
        pool: {
            max: 10,
            min: 0,
            idleTimeoutMillis: 100000,
        },
    }),

Hi, I have error on while streaming 1000.000. It begin streaming normally but after interruption can’t proceed.

Code looks like following:

 const requestGenerator = MssqlAdaptor.Factory(mssql);
            requestGenerator.then((adaptor) => {
                let request = adaptor.request;


                request.on('error', function (err){
                    console.log('Request error');
                    console.log(err);

                });
                request.on('row', function (data){
                    console.log('Next data: ' + data);
                    elasticAdaptor.addRawData([data]);

                });
                request.on('done', function (){
                    elasticAdaptor.replaceOldIndex().then(function (err, result) {
                        done(null, "Finished");

                    });
                });

                adaptor.executeRequest(query);
            });
Request error
{ RequestError: Timeout: Request failed to complete in 50000ms
    at Request.userCallback (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/mssql/lib/tedious.js:802:25)
    at Request.callback (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/request.js:33:27)
    at Connection.message (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/connection.js:1209:31)
    at Connection.dispatchEvent (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/connection.js:519:45)
    at MessageIO.<anonymous> (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/connection.js:439:23)
    at emitNone (events.js:86:13)
    at MessageIO.emit (events.js:185:7)
    at ReadablePacketStream.<anonymous> (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/message-io.js:92:15)
    at emitOne (events.js:96:13)
    at ReadablePacketStream.emit (events.js:188:7)
    at readableAddChunk (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_readable.js:198:18)
    at ReadablePacketStream.Readable.push (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_readable.js:157:10)
    at ReadablePacketStream.Transform.push (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-
[mssql_debug_1486634130964.txt](https://github.com/patriksimek/node-mssql/files/763287/mssql_debug_1486634130964.txt)
[mssql_debug_1486634516673.txt](https://github.com/patriksimek/node-mssql/files/763293/mssql_debug_1486634516673.txt)
[mssql_debug_1486634844538.txt](https://github.com/patriksimek/node-mssql/files/763294/mssql_debug_1486634844538.txt)
[mssql_debug_1486634900132.txt](https://github.com/patriksimek/node-mssql/files/763291/mssql_debug_1486634900132.txt)
[mssql_debug_1486634978027.txt](https://github.com/patriksimek/node-mssql/files/763292/mssql_debug_1486634978027.txt)




stream/lib/_stream_transform.js:123:32)
    at ReadablePacketStream._transform (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/tedious/lib/message-io.js:57:16)
    at ReadablePacketStream.Transform._read (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_transform.js:159:10)
    at ReadablePacketStream.Transform._write (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_transform.js:147:83)
    at doWrite (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_writable.js:313:64)
    at writeOrBuffer (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_writable.js:302:5)
    at ReadablePacketStream.Writable.write (/Users/alse/src/Forks/EyeFitU.NodeJs.Server/node_modules/readable-stream/lib/_stream_writable.js:241:11)
    at CleartextStream.ondata (_stream_readable.js:561:20)
    at emitOne (events.js:96:13)
    at CleartextStream.emit (events.js:188:7)
  name: 'RequestError',
  message: 'Timeout: Request failed to complete in 50000ms',
  code: 'ETIMEOUT',
  number: 'ETIMEOUT',
  lineNumber: undefined,
  state: undefined,
  class: undefined,
  serverName: undefined,
  procName: undefined }

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:6

github_iconTop GitHub Comments

2reactions
munir131commented, May 18, 2017

I think streaming should be free from whatever set in requestTimeout

0reactions
dhensbycommented, Feb 13, 2019

This is most likely because requests didn’t implement any kind of pausing or back pressure. This is now implemented in v5

Read more comments on GitHub >

github_iconTop Results From Across the Web

Firebase Rest API Streaming error: Connection gets closed ...
Firebase Rest API Streaming error: Connection gets closed when child has huge data in it - Stack Overflow. Collectives™ on Stack Overflow – ......
Read more >
Troubleshooting Streaming - Oracle Help Center
This error occurs because the PutMessages call sent to the service is too large. The size must be less than or equal to...
Read more >
Troubleshooting Kinesis Data Streams Consumers
Some Kinesis Data Streams Records are Skipped When Using the Kinesis Client Library. The most common cause of skipped records is an unhandled...
Read more >
80020009 error when you retrieve data - SQL Server
SQL Server is sending back the data on the wire and the client is essentially receiving a stream of bits read sequentially on...
Read more >
Failure on writing large amount of data to excel even ... - GitHub
I am using exceljs with Streams but it still fails to export large amount of data due to Heap out of memory issue....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found