question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Slower Elasticsearch on .Net Core

See original GitHub issue

NEST/Elasticsearch.Net version: 5.4.0 Elasticsearch version: 5.2.2

Description of the problem including expected versus actual behavior: We just updated to .Net core 1.1.1 in production. We run our servers in Debian 8. Our ElasticClient is a singleton for our application. We notice an increase of +120% of the time it takes for the same queries we used to have on .NET Framework 4.5.

For our application, we build a raw elasticsearch string, and we query it like so :

public IEnumerable<T> RawQuery(string query, int limit, params string[] fields)
{
    var client = ElasticsearchConnectionPool.Current.GetClient(_clusterName);
    try
    {
        ISearchResponse<T> result;
        if (fields != null && fields.Any())
        {
            result = client.Search<T>(n => n
                .Index(_index)
                .Size(limit)
                .Source(s => s.Includes(f => f.Fields(fields)))
                .Query(q => q.Bool(f => f.Filter(t => t.Raw(query)))));
        }
        else
        {
            result = client.Search<T>(n => n
                .Index(_index)
                .Size(limit)
                .Query(q => q.Bool(f => f.Filter(t => t.Raw(query)))));
        }

        if (Logger.Current.IsDebugEnabled)
        {
            Log.Debug("ElasticsearchContext.Query", "Debug info", result.DebugInformation);
        }

        if (result.ApiCall.Success)
        {
            return result.Hits.Where(hit => hit.Source != null).Select(hit =>
            {
                hit.Source.Id = Extensions.Deserialize<TIndex>(hit.Id);
                return hit.Source;
            });
        }
        else
        {
            Log.Error("ElasticSearch", "Query failed when api was called", query, result.ApiCall.ToString());
            return Enumerable.Empty<T>();
        }
    }
    finally
    {
        ElasticsearchConnectionPool.Current.ReleaseClient(_clusterName, client);
    }
}

It seems that when our servers are running on windows, the performance is better.

If it helps, we noticed this stacktrace popping from time to time : May 23 12:49:49: ERROR [Elasticsearch.Net.ElasticsearchClientException: Maximum timeout reached while retrying request ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) May 23 12:49:49: at System.Net.Http.HttpClient.<FinishSendAsync>d__58.MoveNext() May 23 12:49:49: --- End of stack trace from previous location where exception was thrown --- May 23 12:49:49: at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) May 23 12:49:49: at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) May 23 12:49:49: at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Connection\HttpConnection-CoreFx.cs:line 70 May 23 12:49:49: --- End of inner exception stack trace --- May 23 12:49:49: at Elasticsearch.Net.RequestPipeline.BadResponse[TReturn](ElasticsearchResponse1& response, RequestData data, List1 pipelineExceptions) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 501 May 23 12:49:49: at Elasticsearch.Net.Transport1.Request[TReturn](HttpMethod method, String path, PostData1 data, IRequestParameters requestParameters) in c:\Projects\elastic\net-5\src\Elasticsearch.Net\Transport\Transport.cs:line 104 May 23 12:49:49: at Nest.LowLevelDispatch.SearchDispatch[T](IRequest1 p, PostData1 body) in c:\Projects\elastic\net-5\src\Nest\_Generated\_LowLevelDispatch.generated.cs:line 2847 May 23 12:49:49: at Nest.ElasticClient.Nest.IHighLevelToLowLevelDispatcher.Dispatch[TRequest,TQueryString,TResponse](TRequest request, Func3 responseGenerator, Func3 dispatch) in c:\Projects\elastic\net-5\src\Nest\ElasticClient.cs:line 57 May 23 12:49:49: at AntVoice.Common.DataAccess.ElasticsearchClient.ElasticsearchContext2.RawQuery(String query, Int32 limit, String[] fields)`

Steps to reproduce:

  1. Run a server on linux
  2. See the increase of timings for the same queries

Provide ConnectionSettings (if relevant):

var pool = new StaticConnectionPool(new Uri("http://elastic01-prod:9200));
_client = new ElasticClient(new ConnectionSettings(pool).ThrowExceptions());

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:15 (8 by maintainers)

github_iconTop GitHub Comments

3reactions
ThomasSkyldahlcommented, Dec 6, 2018

@Mpdreamz the new sockets implementation has now been released, is there anything we can do to get the ball rolling on the new IConnection implementation?

3reactions
ejsmithcommented, Mar 14, 2018

Have you seen the changes that they have made to HttpClient and the new IHttpClientFactory in .net core 2.1? Apparently they changes the handler to use the new managed sockets implementation and saw a massive performance increase.

https://blogs.msdn.microsoft.com/webdev/2018/02/28/asp-net-core-2-1-preview1-introducing-httpclient-factory/

https://github.com/dotnet/corefx/issues/23401

Read more comments on GitHub >

github_iconTop Results From Across the Web

Elastic search Nest Client is very slow - asp.net core
I have build Elasticsearch repository in asp.net core 3.1. Settings of Client var settings = new ConnectionSettings(new Uri(url)) .
Read more >
finding and fixing slow Elasticsearch queries
Common causes of slow Elasticsearch queries ... Queries are targeting too many shards, exceeding the number of cores in the cluster.
Read more >
Usage recommendations | Elasticsearch .NET Client [8.9]
Access to Elasticsearch is sometimes slow or delayed, and some complex queries may take several seconds to return. If such operations are blocked...
Read more >
Tune for indexing speed | Elasticsearch Guide [8.9]
First try to index 100 documents at once, then 200, then 400, etc. doubling the number of documents in a bulk request in...
Read more >
Sparse slow NEST Searches - Elasticsearch
Hi everyone, We are using ElasticSearch Nest client to build an API with Dot Net Core to run queries towards an ElasticSearch cluster....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found