question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Paged Queries in Parallel

See original GitHub issue

If you do multiple queries against the same client (or client connection pool) with paged: true the two searches seem to corrupt one another resulting in

ProtocolError: paged results cookie is invalid

As long as you limit yourself to one search at a time you’re ok.

Am I doing something wrong? Is this expected behaviour? I need to know if I need to abandon the persistent connection pool and build/tear down a connection for every search.

  var _LDAPSearch = function (opts) {
    return new Promise(function (resolve, reject) {
      var client = opts.client;
      var base = opts.base;
      var options = opts.options;

      return client.search(base, options, function (err, res) {
        if (err) return reject(err);

        var entries = [];

        res.on('searchEntry', function (entry) {
          return entries.push(entry);
        });
        res.on('error', function (err) {
          if (err) sails.log.warn(err);
        });
        res.on('end', function (result) {
          if (result.errorMessage) return reject(result.errorMessage);

          return resolve(entries);
        });
      });
    });
  };

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
pfmooneycommented, Oct 9, 2015

I suspect that the LDAP server you’re communicating with does not support concurrent paged requests for the same search criteria on the same connection. Paged searches in LDAP function by passing an opaque cookie back and forth between the server and the client. This cookie identifies the specific search to the server, allowing it to continue serving results where it left off in the search. Since you have two searches for the exact same thing going on simultaneously, it could be that the server believes the cookie(s) you’re passing back are invalid.

You will likely need to ensure, via your own custom logic, that two identical paged searches do not occur on the the same client connection simultaneously.

0reactions
jvanalstcommented, Oct 9, 2015

Alright, good to know. I’ll abandon the persistent connection pool and just build/teardown a client for every query.

Read more comments on GitHub >

github_iconTop Results From Across the Web

What's good about offset pagination; designing parallel cursor ...
Running multiple paginated queries in parallel is essentially circumventing the API provider's intent to limit the number of items requested ...
Read more >
ElasticSearch Parallel Pagination by Kafka | by Jay Ehsaniara
This process is sequentially asynchronous, which means to get through 1,000,000 documents you need to call ES 1,000,000/10,000=100 times one ...
Read more >
How to fetch parallel pagination data from big query
I am fetching paginated data from bq since data ...
Read more >
What's good about offset pagination; designing parallel cursor ...
Cursor-based pagination is difficult for a client to parallelize because it can't know what cursor to send for the next page until it's...
Read more >
Parallel Queries | TanStack Query Docs
"Parallel" queries are queries that are executed in parallel, or at the same time so as to maximize fetching concurrency.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found