Stable JSON stringify to avoid cache misses

It looks like this lib is using JSON.stringify on requests. Per the elasticsearch query cache docs:

if the JSON changes — for instance if keys are output in a different order — then the cache key will not be recognised.

This means that if we make requests to the elasticsearch client with objects where the keys are not created in the exact same order, we can’t take advantage of the request cache. My unstanding is that these two would not be the same from the cache’s perspective:

await client.search({
  index: 'twitter',
  type: 'tweets',
  body: {
    query: {
      match: {
        body: 'elasticsearch'
      }
    }
  }
})

vs

await client.search({
  type: 'tweets',
  index: 'twitter',
  body: {
    query: {
      match: {
        body: 'elasticsearch'
      }
    }
  }
})

If we switch to any of these libs, we could make sure the request cache is hit more often. If it’s much slower, maybe we could make it a config option for the client (e.g. stableStringify: true)?

[spalger]

Yeah, that could happen, but in your example the cache would still be used since only the keys within the body are relevant.

If you want to try out one of these libs you can specify a custom serializer in the config like so:

import DefaultJsonSerializer from 'elasticsearch/src/lib/serializers/json';
import jsonStableStringify from 'json-stable-stringify';

class CustomSerializer extends DefaultJsonSerializer {
  serialize(val, replacer, space) {
    return jsonStableStringify(val, { replacer, space })
  }
}

CustomSerializer.prototype.serialize.contentType = 'application/json';

const client = new elasticsearch.Client({
  serializer: CustomSerializer
})

Interested to hear what your experience is, if the potential overhead of a slower serializer is beneficial in the long run for potentially faster querying. I think it would be pretty surprising for the keys JSON bodies from a single application that are requesting the same thing to not always be sent in the same order, mostly because I expect that people won’t be copy/pasting the same query in different places, but rather each place they query sends unique queries and the function body produced by a single function probably creates a specific query the same way every time.

I can imagine how that might not be the case for contexture, so maybe an option like a custom serializer is more appropriate if it does slow down serialization in a meaningful way.

[daedalus28]

Sorry for the delay - I’ve opened PR #727 adding the relevant documentation.