[BUG] ApplicationTokenProvider should reuse HttpClientHandler instead of creating it on every call
See original GitHub issueLibrary name and version
Microsoft.Rest.ClientRuntime.Azure.Authentication 2.4.1
Describe the bug
Problem
Every call to ApplicationTokenProvider.LoginSilentAsync
creates a new HttpClientHandler
that will only be used only for the duration of the call.
This can easily lead to socket/port exhaustion.
The issue frequently creating and disposing HttpClientHandler
explained in detail in the official Microsoft docs
They mention HttpClient
, but essentially the issue is with the HttpClientHandler
that is created in HttpClient
default constructor
Though this class implements IDisposable, declaring and instantiating it within a using statement is not preferred because when the HttpClient object gets disposed of, the underlying socket is not immediately released, which can lead to a socket exhaustion problem. For more information about this issue, see the blog post You’re using HttpClient wrong and it’s destabilizing your software.
After HttpClientHandler
is disposed, the underlaying socket connections are still active for some time.
This issue caused (or contributed to) socket exhaustion on one of our production instances. I also found a related unanswered stackoverlfow issue describing exactly that problem.
Proposed solution
Most of the other azure sdk libraries provide a way to pass your own HttpClient
. In our case, we use the default HttpClientFactory
from ASP.NET that managers the lifetime of HttpClientHandler
and clients.
There should be a way to pass HttpClient
or HttpManager
or something like that to ApplicationTokenProvider.LoginSilentAsync
call.
The currently referenced version of Microsoft.IdentityModel.Clients.ActiveDirectory
(4.3.0) does not support using IHttpClientFactory, the creation of HttpClientHandler
is hard coded
However, an option to pass IHttpClientFactory
was introduced from version 5.0.1-preview
Here’s a link to the instructions on how to use IHttpClientFactory
with ADAL:
IHttpClientFactory myHttpClientFactory = new MyHttpClientFactory();
AuthenticationContext authenticationContext = new AuthenticationContext(
authority: "https://login.microsoftonline.com/common",
validateAuthority: true,
tokenCache: <some token cache>,
httpClientFactory: myHttpClientFactory);
That probably means changes to the following code and its callers to be able to pass IHttpClientFactory
.
https://github.com/Azure/azure-sdk-for-net/blob/f9858eac647b7f48bf63c8fb87defa656e759147/sdk/mgmtcommon/Auth/Az.Auth/Az.Authentication/ApplicationTokenProvider.cs#L678-L687
Workaround
I think I can replicate the behaviour of this ApplicationTokenProvider.LoginSilentAsync
overload in our codebase to be able to pass HttpClientFactory
to AuthenticationContext
, but it should be fixed at the package level.
Expected behavior
It should be possible to pass HttpClient
to ApplicationTokenProvider.LoginSilentAsync
or to configure ApplicationTokenProvider
to reuse HttpClientHandler
to prevent socket exhaustion
Actual behavior
It should not possible to pass HttpClient
to ApplicationTokenProvider.LoginSilentAsync
or to configure ApplicationTokenProvider
to reuse HttpClientHandler
to prevent socket exhaustion.
And it caused (or contributed to) a socket exhaustion in our production instance
Reproduction Steps
Here’s an example Program.cs
that runs LoginSilentAsync
in a loop and reports the count of active Tcp connections before and after each call.
Please note, the count always increases, the connections are not even released on application stop.
Program.cs
using System;
using System.Threading.Tasks;
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.Rest.Azure.Authentication;
namespace HttpClientTestingTest
{
class Program
{
static async Task Main(string[] args)
{
var clientCredential = new ClientCredential("123", "345");
for (var i = 0; i < 10; i++)
{
try
{
ReportConnectionCount($"Before {i}");
await ApplicationTokenProvider.LoginSilentAsync("http://localhost/test", clientCredential, ActiveDirectoryServiceSettings.Azure);
}
catch
{
ReportConnectionCount($"After {i}");
}
}
}
static void ReportConnectionCount(string info)
{
var properties = System.Net.NetworkInformation.IPGlobalProperties.GetIPGlobalProperties();
Console.WriteLine($"ConnectionCount: {properties.GetActiveTcpConnections().Length} - {info}");
}
}
}
Environment
The production issue was happening in Azure App Service The reproduction on MacOS using netcore 3.1 and .NET 5.
Issue Analytics
- State:
- Created 2 years ago
- Comments:10 (6 by maintainers)
Top GitHub Comments
Thanks, @anaismiller, and apologies for the confusion. Assigning to @ArthurMa1978 for management library triage.
Thank you for your feedback. Tagging and routing to the team members best able to assist.