DI-related instability when using HttpClientFactory

Investigative information

• Timestamp: February 3rd, 2020, ~5:40PM CST
• Function App version: ~3, Runtime version: 3.0.13107
• Function App name: (will provide if needed)
• Function name(s) (as appropriate): (will provide if needed)
• Invocation ID: (will provide if needed)
• Region: Central US

Repro steps

I’m referencing the following nuget packages:

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.0.0" />
    <PackageReference Include="Microsoft.Data.SqlClient" Version="1.1.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="3.1.1" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.Extensions.Http" Version="3.1.1" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.3" />
  </ItemGroup>

I’m using Microsoft.Azure.Functions.Extensions for DI like so:

    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddHttpClient();
            builder.Services.AddScoped<IApiService, ApiService>();
        }
    }

And the service looks something like this:

    public class ApiService : IApiService
    {
        private readonly IHttpClientFactory _httpClientFactory;

        public ApiService(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
        }

        public void MakeRequest()
        {
            var httpClient = _httpClientFactory.CreateClient();
            httpClient.PostAsync(...);
        }
    }

And the function looks something like this:

    public class MyTrigger
    {
        private readonly IApiService _apiService;

        public MyTrigger(IApiService apiService)
        {
            _apiService = apiService;
        }

        [FunctionName("MyTrigger")]
        public void Run([TimerTrigger("0 */5 * * * *", RunOnStartup = true)]TimerInfo timerInfo, CancellationToken cancellationToken)
        {
            foreach (var x in alot)
            {
                _apiService.MakeRequest();
            }
        }
    }

Here’s the error message and stack trace I see in App Insights:

Scope disposed{no name, Parent=disposed{no name, Parent={no name, Parent={no name}}}} is disposed and scoped instances are disposed and no longer available.

DryIoc.ContainerException:
   at DryIoc.Throw.It (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 8990)
   at DryIoc.Scope.TryGet (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 7880)
   at DryIoc.Container+InstanceFactory.GetAndUnwrapOrDefault (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 1479)
   at DryIoc.Container+InstanceFactory.GetInstanceFromScopeChainOrSingletons (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 1465)
   at DryIoc.Container.DryIoc.IResolver.Resolve (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 307)
   at lambda_method (Anonymously Hosted DynamicMethods Assembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 223)
   at DryIoc.Container.DryIoc.IResolver.Resolve (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 194)
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.ScopedServiceProvider.GetService (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\ScopedServiceProvider.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 25)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService (Microsoft.Extensions.DependencyInjection.Abstractions, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService (Microsoft.Extensions.DependencyInjection.Abstractions, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateHandlerEntry (Microsoft.Extensions.Http, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.Extensions.Http.DefaultHttpClientFactory+<>c__DisplayClass14_0.<.ctor>b__1 (Microsoft.Extensions.Http, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Lazy`1.ViaFactory (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Lazy`1.ExecutionAndPublication (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Lazy`1.CreateValue (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Lazy`1.get_Value (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateHandler (Microsoft.Extensions.Http, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateClient (Microsoft.Extensions.Http, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Net.Http.HttpClientFactoryExtensions.CreateClient (Microsoft.Extensions.Http, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)

Expected behavior

I expect HttpClientFactory to be supported and stable for Azure Functions v3.

Actual behavior

It went well for a good 1k+ requests, but eventually it bombed out with essentially the same error as seen in #5060. Unlike that issue however, I’m injecting HttpClientFactory instead of HttpClient like was recommended in the comments and calling factory.CreateClient() each time a request needs to be made. Looks like #5074 may have been a reproduction of this as well. Is there something else I should be doing in the code to avoid this bizarre issue?

Known workarounds

I switched back to the old school approach of a static HttpClient and that works well enough for my needs, but I’m assuming HttpClientFactory is the way of the future.

[a-vishar]

Any update on this? There are numerous issues which are the same as this one, a couple I found with a quick search:

https://github.com/Azure/azure-functions-host/issues/5851 https://github.com/Azure/azure-functions-host/issues/6087

6 months of DI issues for something as fundamental as a Http Client without an official word on the subject isn’t great, could someone at least respond?

[PhilippK13]

Just stumbled upon the same issue(azure functions v3, .net core 3.1).

We had a very similar setup: the function was making a call to service but was not awaiting it. Something like this(adjusted for brevity):

public class EventGridTrigger
 {
        private readonly IProcessor processor;

        [FunctionName(nameof(EventGridTrigger))]
        public async Task Run(...)
        {
               this.processor.ProcessAsync(...);
        }
 }
...

public class Processor: IProcessor
{
    private readonly IHttpClientFactory httpClientFactory;

    public async Task ProcessAsync(...)
    {
        var client = this.httpClientFactory.CreateClient();
        var response = await client.PostAsync(...);
    }
}

Note that the ProcessAsync call is not awaited and the HttpClient is created inside the ProcessAsync method.

My working theory is that the scoped service provider gets injected here , so what happens is that because we are not awaiting the ProcessAsync the scope gets disposed, and when CreateClient tries to get the HttpMessageHandlerBuilder, here, it is already too late and causes the “Scope disposed…” exception.

To solve the issue, we simply can await the ProcessAsync call. Alternatively, the Processor class can be rewritten to instantiate HttpClient directly in the constructor, like so:

public class Processor: IProcessor
{
    private readonly IHttpClientFactory httpClientFactory;

    public Processor(IHttpClientFactory httpClientFactory)
    {
        this.httpClient = httpClientFactory.CreateClient();
    }

    public async Task ProcessAsync(...)
    {
        var response = await this.client.PostAsync(...);
    }
}

The second approach, of course, depends on how and when the Processor is instantiated. It can be instantiated lazily causing the same issue.

One thing to mention is that we noticed this issue in EventGridTrigger azure function. I was not able to reproduce this locally on an HttpTrigger, though I’m running functions runtime v4 locally, so maybe this is fixed in v4?

Would be great for someone from Azure Functions team to take a look at this issue. @brettsam maybe you can help?