Cookies being set twice with BrowserContext request

The following code:

var configuration = Configuration.Default.WithDefaultLoader().WithCookies();
var cookieProvider = configuration.Services.OfType<ICookieProvider>().First();
cookieProvider.SetCookie("https://subdomain.my-fully-qualified.url", "foo=bar,baz=quux");
var document = await BrowsingContext.New(configuration).OpenAsync("https://subdomain.my-fully-qualified.url/resource");

causes the cookies foo and baz to be set twice in the request to https://subdomain.my-fully-qualified.url/resource.

I’m using AngleSharp 0.9.9 on .NET Core 2.0 (via the official NuGet package).

Is this a bug, and if not, what is the correct way to set cookies on a BrowsingContext request?

[IanKemp]

Okay, first of all my example was incorrect/incomplete: this issue only occurs for requests to subdomains. The example has been updated to reflect what I’m actually doing.

Secondly, this issue is not fixed in devel, but I can tell you how and why it occurs. The bug lies within the RequestState class in DefaultHttpRequester.cs, in particular the constructor call to SetHeaders() followed by SetCookies():

public RequestState(Request request, IDictionary<String, String> headers)
{
    _cookies = new CookieContainer();
    _headers = headers;
    _request = request;
    _http = WebRequest.Create(request.Address) as HttpWebRequest;
    _http.CookieContainer = _cookies;
    _http.Method = request.Method.ToString().ToUpperInvariant();
    _buffer = new Byte[BufferSize];
    SetHeaders(); // including cookies!
    SetCookies(); // cookies again...
    AllowCompression();
    DisableAutoRedirect();
}

SetHeaders() copies all relevant HTTP headers from the AngleSharp.Io.Request object passed into the constructor to the HttpWebRequest to be dispatched… if these headers contain the Cookie header, it will be copied too. Then, SetCookies() runs and explicitly copies only the Cookie header from the AngleSharp.Io.Request into the HttpWebRequest.CookieContainer.

End result, when the HttpWebRequest is actually executed, the cookies from its CookieContainer are merged with the cookies in its Headers - causing the duplication. At first glance this would seem like a bug in HttpWebRequest, but I’d disagree, because that class is a (relatively) low-level object that doesn’t do handholding (or in this case, cookie disambiguation).

It seems to me that the simplest way to fix this would be to alter SetHeaders to explicitly ignore the Cookie header, when copying headers from AngleSharp.Io.Request to HttpWebRequest.

[FlorianRappl]

Hi @IanKemp - sounds awesome. Yes, please file a bug with them (pointing to this issue). However, since .NET Core 2 is already released we still need to fix it in AngleSharp in my opinion.

Thanks also for your efforts to confirm my guess 🍺. Really nicely done and much appreciated 🥇 !