[Azure Search] NullReferenceException when checking if index exists

(This issue is based on my question on Stack Overflow)

When trying to see if an index exists for a non-existing index with the .NET SDK (both 3.0.4 and 4.0.0-preview) the ExistsAsync (as well as Exists, and ExistsWithHttpMessagesAsync) throws the following exception.

System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Azure.Search.IndexesOperations.<GetWithHttpMessagesAsync>d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Search.ExistsHelper.<ExistsFromGetResponse>d__0`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Search.IndexesOperationsExtensions.<ExistsAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at InphizCore.MiddleLayer.Services.AzureSearch.Services.AzureSearchWriter`1.<AssertIndexExists>d__8.MoveNext() in C:\xxx\AzureSearchWriter.cs:line 109

Using http rest from postman works fine and returns a message saying the index doesn’t exists.

 public async Task AssertIndexExists()
        {
            try
            {
                if (await searchServiceClient.Indexes.ExistsAsync(options.Value.IndexName) == false)
                {
                    searchServiceClient.Indexes.Create(new Index(options.Value.IndexName, FieldBuilder.BuildForType<SearchableItemModel>(), corsOptions: new CorsOptions(new List<string> { "*" })));
                }
            }
            catch (Exception e)
            {
                logger.LogError($"Azure Search Index '{options.Value.IndexName}' could not be created. ({e.Message})");
                throw e;
            }
        }

This is how the client looks when running in a Unit Test:

This is how it looks from AspNetCore MVC:

Is seems like the Search Client fails to create instances of FirstMessageHandler, HttpClient and HttpClientHandler. Why doesn’t it throw?

Well, this is strange but possibly some inner workings of the SearchServiceClient.

This doesn’t work:

If I add a HttpClientHandler which I pass along to another of the constructors AND set lifetime to Singleton or Transient it works. I previously had Scoped.

 services.AddTransient<SearchServiceClient>(x =>
 {
     var httpClientHandler = new HttpClientHandler();

     var options = x.GetRequiredService<IOptions<AzureSearchOptions>>();
     var client = new SearchServiceClient(options.Value.SearchServiceName, new SearchCredentials(options.Value.AdminApiKey), httpClientHandler);
     return client;
 });

UPDATE: This works:

Behavior in solution above was unpredictable and I ended up with this method in the calling class.

When I created a new instance for each time I need a client it doesn’t seem to give any errors at all:

  protected SearchServiceClient GetAzureSearchServiceClient()
        {
            return new SearchServiceClient(options.Value.SearchServiceName, new SearchCredentials(options.Value.AdminApiKey));
        }

[brjohnstmsft]

@jstensved Looking at the code, I see that the FirstMessageHandler, HttpClient, and HttpClientHandler properties are all defined in the ServiceClient base class. By tracing through the constructor logic in the base class and in SearchServiceClient, I can see that some subset of these properties is always initialized to non-null values. However, it is possible for all three to be null at the same time if Dispose was called prematurely. The fact that you observed different behavior for different DI container lifetimes seems to support this hypothesis.

As a next step to reproducing this problem, I recommend that you substitute your own class that implements IDisposable into your ASP.NET Core DI setup code. Then you can put a breakpoint on Dispose and see where it’s being called from and when.

In terms of how best to create a SearchServiceClient, I’d recommend using the Singleton lifetime, or register a singleton instance. This is because each SearchServiceClient has its own HttpClient, and creating too many of those can exhaust the TCP connection pool, leading to performance issues as the number of requests to your application increases. Just be careful not to call any methods or access any properties on SearchServiceClient outside of your factory method unless they’re thread-safe (for example, don’t set any settable properties). This advice goes for SearchIndexClient too.

Please let me know if you manage to confirm whether a premature call to Dispose is causing the problem.

[brjohnstmsft]

I haven’t heard back from @jstensved so I’m closing this since I can’t reproduce it.