Sometimes ensureIndexes fails

UPDATE (TL;DR)

For those searching this issue and looking for a quick fix: a forked and patched version of ottoman is available at https://github.com/zachlankton/node-ottoman/tree/610-ensure-fix.
This will only be maintained until the patch is published in the official repo. ( Currently pending PR: #611 )

It can also be installed with NPM:

npm install ottoman@npm:@zachwritescode/ottoman

Alternatively, you can update your package.json dependencies with:

"ottoman": "npm:@zachwritescode/ottoman@^2.0.1",

Original Post:

Hello, I am experiencing an intermittent error.

Ensuring Indexes...
[Error: LCB_ERR_KEYSPACE_NOT_FOUND (404): Keyspace is not found (collection or bucket does not exist)] {
  code: 404,
  ctxtype: 'query',
  first_error_code: 12003,
  first_error_message: 'Keyspace not found in CB datastore: default:connext._default.UserProfile',
  statement: 'BUILD INDEX ON `connext`.`_default`.`UserProfile`(`Ottoman_defaultUserProfile`) USING GSI',
  client_context_id: 'd9c1ccc274582b8e',
  parameters: '',
  http_response_code: 500,
  http_response_body: ''
}

This only happens sometimes when calling the await start() stand alone function. However, this error never occurs when I do the following:

await ottoman.connect(cnx)
await ottoman.ensureCollections()
await pause(1000); //Helper function to force delays
await ottoman.ensureIndexes();

I’ve written a test script that reproduces the problem pretty reliably ( strange for an intermittent problem 😊 ) It also runs a separate set of tests using the above solution as well as using the couchbase node.js SDK directly with no pause and also with a pause between creating the collection and creating the index.

It appears the main issue is needing a delay between creating collection and creating indexes, regardless if done through ottoman or couchbase node sdk?

You can get the test script here https://gist.github.com/zachlankton/aaaa7d6d1cc1ee34e0974bc1ee05b191

Here is a snapshot of the results I’m getting from this script on my machine:

PS C:\Users\zachl\Documents\ottoman-test> node ottoman-test.mjs

Testing Ottoman with Start....
Had 5 errors out of 10 tests 

Testing Ottoman with Pause....
Had 0 errors out of 10 tests 

Testing with Couchbase....
Had 7 errors out of 10 tests 

Testing with Couchbase Pause...
Had 0 errors out of 10 tests 

[zachlankton]

Thanks @AV25242 … I believe there was some misunderstanding here. The error LCB_ERR_KEYSPACE_NOT_FOUND was not handled in a way that would provide for what you describe. The error persisted even with the ignoreWatchIndexes set to true and hooking into the callback. This is because the collection was not ready, so the index could not be built.

I cloned the repo and ran the test suite on the couchbase/server-sandbox database image and all tests pass wonderfully in legacy tests! I see that the non legacy tests fail for this same error that I am reporting (as well as query Planning Failures most likely due to the index not being ready)

I modified the tryCreateCollection logic to guarantee the collection is ready and fixed the first 6 failing tests for the LCB_ERR_KEYSPACE_NOT_FOUND.

I reviewed the code and found that the “if” block for ignoreWatchIndexes will ONLY run if the indexOnlinePromise is defined… and indexOnlinePromise would only get defined in ensure_n1ql_indexes.ts IF there were indexReadyHooks registered. I refactored the logic so that ensureIndexes will return the indexOnlinePromise as it should.

Now all Legacy and non Legacy Tests pass perfectly!

I will have a pull request prepared shortly.

[ejscribner]

Hey @zachlankton! Wanted to update you that this has been added in the latest release of Ottoman (v2.1.0), available now on NPM. Thanks for the support!