How to create HttpClientHandler later than configuration time

As dicussed briefly in #71:

Looking at the usage of services.AddHttpClient().ConfigurePrimaryHttpMessageHandler(), I’m wondering how to configure it when the necessary information is not available at configuration time.

Say that each time we want to use the HttpClient, we load a certificate (perhaps based on a fingerprint), with the intention of getting an HttpClientHandler that uses that particular certificate. Such information may not be available (or even remotely convenient) at configuration time, let alone for each endpoint.

How can we get a properly managed HttpClientHandler at a later stage, when we have all the information (including potential certificates) about a particular endpoint we are connecting to?

Note that we want the client handler’s lifetime to be handled for us: don’t create a new one for every request, to avoid opening tons of TCP connections, but don’t keep one around indefinitely either, because DNS changes must be accounted for.

[tsaluszewski]

We are having a similar issue, we are utilizing the IHttpClientFactory to avoid opening unnecessary TCP connections (we had this issue before with per request created client).

Our requirement is to use Digest authentication when the actual credentials should be set at run-time at each request rather at configure time. See below:

• DI services configuration

public static ServiceCollection CreateServices()
 {
         var serviceCollection = new ServiceCollection();
         serviceCollection.AddHttpClient("WithDigestAuthentication")
                .ConfigurePrimaryHttpMessageHandler(() => new DigestAuthenticationHandler());
         return serviceCollection;
 }

• Custom message handler

public class DigestAuthenticationHandler : HttpClientHandler
    {
        public const string Username = "Digest.Username";
        public const string Password = "Digest.Password";

        private const string AuthType = "Digest";

        protected override async Task<HttpResponseMessage> SendAsync(
            HttpRequestMessage request,
            CancellationToken cancellationToken)
        {
            // Nasty hack to get around
            // "Properties can only be modified before sending the first request." error :(
            if (this.Credentials == null)
            {
                var credCache = new CredentialCache();
                var networkCredential = new NetworkCredential(
                    request.Properties[Username].ToString(),
                    request.Properties[Password].ToString());

                credCache.Add(request.RequestUri, AuthType, networkCredential);
                this.Credentials = credCache;
            }

            return await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
        }
    }

• Code that adds actual credentials to the HttpRequestMessage.Properties as advised in Make HTTP requests using IHttpClientFactory in ASP.NET Core

        private HttpRequestMessage WithCredentials(HttpRequestMessage request)
        {
            request.Properties[DigestAuthenticationHandler.Username] = this.username;
            request.Properties[DigestAuthenticationHandler.Password] = this.password;
            return request;
        }

The problem is that once HttpMessageHandler.Credentials are set these cannot be re-set on a subsequent request.

Can you please advise how to overcome this issue? We thought about creating a pool of preconfigured “WithDigestAuthenticationXYZ” HttpClients and selecting one at runtime via our custom client factory method that would check whether given client Credentials are equal. When the pool is drained out we can revert to just returning a new instance of HttpClient. What do you think?

[rynowak]

To avoid socket exhaustion, IHttpClientFactory is a necessity

Can someone summarize for me why this is believed to be the case? Is it because a ton of distinct components all want to share the same instance and can’t otherwise?

@stephentoub

Keeping an instance around forever wasn’t a good solution in 2.0 when this feature was added because you’d end up with stale DNS. Keep in mind that we’re talking about environments where you keep HTTP connections open for days.

Now that we have SocketsHttpHandler you can do this and still avoid the stale-DNS problem by configuring PooledConnectionLifetime. However, we haven’t done enough to get the word out, so people are still convinced that the client factory is the only solution. The property has no documentation at all 😢 I just sent a PR to suggest this in the client factory docs https://github.com/aspnet/AspNetCore.Docs/pull/15854

---

• HttpClientHandler/SocketsHttpClientHandler associate authN/proxy/network related settings with the handler instance
• IHttpClientFactory is a framework for configuring the outgoing http stack
• IHttpClientFactory also encourages reuse of the underlying handlers These three points are in conflict 😆

Would you please explain more details? For me, reusing handlers with IHttpClientFactory should be a must. The users shouldn’t worry about the underlying problems such as ‘Socket Exhaustion’.

@LeaFrock

Sure.

A client handler (SocketsClientHandler) is for all intents and purposes a connection pool. It represents a set of a connections to a set of hosts. This is why keeping a long-lived one, or reusing/sharing one is efficient, because you can share the connections you don’t have to wait for them to be reestablish.

Features like client certificates or proxy settings describe how those connections are made. You cannot share the same connections to the same host with two different client certificates - you’re asking to do something impossible.

Since in .NET a client handler represents a pool of connections, that is where settings related to client certificates and proxy servers life. You could argue that it’s inconvenient for you, but that’s the design .NET has.

So back to my three statements.

• Users want to multiplex proxies and certificates (including multiple certs and proxies to the same hosts)
• They want this to be easy to code up (using client factory)
• They want this to be efficient (reuse connections where possible)

These desires conflict with how all of this is designed.