Slower Elasticsearch on .Net Core

NEST/Elasticsearch.Net version: 5.4.0 Elasticsearch version: 5.2.2

Description of the problem including expected versus actual behavior: We just updated to .Net core 1.1.1 in production. We run our servers in Debian 8. Our ElasticClient is a singleton for our application. We notice an increase of +120% of the time it takes for the same queries we used to have on .NET Framework 4.5.

For our application, we build a raw elasticsearch string, and we query it like so :

public IEnumerable<T> RawQuery(string query, int limit, params string[] fields)
{
    var client = ElasticsearchConnectionPool.Current.GetClient(_clusterName);
    try
    {
        ISearchResponse<T> result;
        if (fields != null && fields.Any())
        {
            result = client.Search<T>(n => n
                .Index(_index)
                .Size(limit)
                .Source(s => s.Includes(f => f.Fields(fields)))
                .Query(q => q.Bool(f => f.Filter(t => t.Raw(query)))));
        }
        else
        {
            result = client.Search<T>(n => n
                .Index(_index)
                .Size(limit)
                .Query(q => q.Bool(f => f.Filter(t => t.Raw(query)))));
        }

        if (Logger.Current.IsDebugEnabled)
        {
            Log.Debug("ElasticsearchContext.Query", "Debug info", result.DebugInformation);
        }

        if (result.ApiCall.Success)
        {
            return result.Hits.Where(hit => hit.Source != null).Select(hit =>
            {
                hit.Source.Id = Extensions.Deserialize<TIndex>(hit.Id);
                return hit.Source;
            });
        }
        else
        {
            Log.Error("ElasticSearch", "Query failed when api was called", query, result.ApiCall.ToString());
            return Enumerable.Empty<T>();
        }
    }
    finally
    {
        ElasticsearchConnectionPool.Current.ReleaseClient(_clusterName, client);
    }
}

It seems that when our servers are running on windows, the performance is better.

If it helps, we noticed this stacktrace popping from time to time : May 23 12:49:49: ERROR [Elasticsearch.Net.ElasticsearchClientException: Maximum timeout reached while retrying request ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) May 23 12:49:49: at System.Net.Http.HttpClient.<FinishSendAsync>d__58.MoveNext() May 23 12:49:49: --- End of stack trace from previous location where exception was thrown --- May 23 12:49:49: at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) May 23 12:49:49: at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Connection\HttpConnection-CoreFx.cs:line 70 May 23 12:49:49: --- End of inner exception stack trace --- May 23 12:49:49: at Elasticsearch.Net.RequestPipeline.BadResponse[TReturn](ElasticsearchResponse1& response, RequestData data, List1 pipelineExceptions) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 501 May 23 12:49:49: at Elasticsearch.Net.Transport1.Request[TReturn](HttpMethod method, String path, PostData1 data, IRequestParameters requestParameters) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Transport\Transport.cs:line 104 May 23 12:49:49: at Nest.LowLevelDispatch.SearchDispatch[T](IRequest1 p, PostData1 body) in c:\Projects\elastic\net-5\src\Nest\_Generated\_LowLevelDispatch.generated.cs:line 2847 May 23 12:49:49: at Nest.ElasticClient.Nest.IHighLevelToLowLevelDispatcher.Dispatch[TRequest,TQueryString,TResponse](TRequest request, Func3 responseGenerator, Func3 dispatch) in c:\Projects\elastic\net-5\src\Nest\ElasticClient.cs:line 57 May 23 12:49:49: at AntVoice.Common.DataAccess.ElasticsearchClient.ElasticsearchContext2.RawQuery(String query, Int32 limit, String[] fields)`

Steps to reproduce:

1. Run a server on linux
2. See the increase of timings for the same queries

Provide ConnectionSettings (if relevant):

var pool = new StaticConnectionPool(new Uri("http://elastic01-prod:9200));
_client = new ElasticClient(new ConnectionSettings(pool).ThrowExceptions());

[ThomasSkyldahl]

@Mpdreamz the new sockets implementation has now been released, is there anything we can do to get the ball rolling on the new IConnection implementation?

[ejsmith]

Have you seen the changes that they have made to HttpClient and the new IHttpClientFactory in .net core 2.1? Apparently they changes the handler to use the new managed sockets implementation and saw a massive performance increase.

https://blogs.msdn.microsoft.com/webdev/2018/02/28/asp-net-core-2-1-preview1-introducing-httpclient-factory/

https://github.com/dotnet/corefx/issues/23401