Performance hit in NEST 6.x

NEST/Elasticsearch.Net version: 5.6.2 6.1.0

Elasticsearch version: 5.5.1 6.2.3

Description of the problem including expected versus actual behavior: In the process of upgrading our Elasticsearch setup from version 5.x to 6.x we have noticed a performance impact that translates into slower response time in our service that uses Elasticsearch as data storage. The slower response is mostly visible when dealing with large documents with complex structures e.g. geojson.

The two Elasticsearch clusters performs equally when we are testing them directly via Apache ab so I guess the performance hit is related to the NEST library.

I have done some performance testing using BenchmarkDotNet (on a simplified implementation described further down) where NEST v. 6 has a noticeable higher response compared to v. 5. I hope you can shed some light on why we are seeing this behaviour. Thanks

Method	Mean	StdDev
NEST 5	73.11 ms	2.558 ms
NEST 6	92.15 ms	3.054 ms

Steps to reproduce:

1. Add an index to Elasticsearch with this mapping

{
  "properties": {
    "boundaries": {
      "properties": {
        "boundary": {
          "type": "geo_shape"
        },
        "type": {
          "type": "text",
          "index": false
        }
      }
    }
  }
}

1. Insert document https://gist.github.com/kaaresylow/ece0d2dd88175653fb4fa44710297d14

var connectionPool = new SingleNodeConnectionPool(new Uri("url"));
var settings = new ConnectionSettings(connectionPool).EnableHttpCompression();
var client = new ElasticClient(settings);

var response = client.Get<Document>(1, idx => idx
    .Index("documents")
    .Type("documentmodel"));

public class Document
    {
        public IEnumerable<PlaceBoundary> Boundaries { get; set; }
    }

    public class PlaceBoundary
    {
        public string Type { get; set; }

        public GeoJsonBoundary Boundary { get; set; }
    }

    public class GeoJsonBoundary
    {
        public string Type { get; set; }

        public object Coordinates { get; set; }

        public object Bbox { get; set; }

        public object Crs { get; set; }
    }

[Mpdreamz]

Just a quick heads up that we expect to switch to Utf8Json as the default internal serializer with 7.0.

One nice thing that @russcam found is that the hand off of the stream to the user defined serializer is now much much faster.

[codebrain]

7.0 now uses Utf8Json as the serialiser, as detailed on the GA release blog post: https://www.elastic.co/blog/nest-and-elasticsearch-net-7-0-now-ga