Cookies aren't always sent to subdomains

Manually adding cookies to the using any of the constructors of Cookie except for this one https://github.com/HtmlUnit/htmlunit/blob/e1f559f15a682a325a55ae4ed41b116653f21eeb/src/main/java/com/gargoylesoftware/htmlunit/util/Cookie.java#L99-L101

results in cookies not being sent for subdomains, even though they should be sent as per section 5.1.3 of RFC 6265.

The root cause is that HtmlUnitDomainHandler delegates to BasicDomainHandler which contains the following code:

// ...
if (cookie instanceof ClientCookie) {
  if (((ClientCookie) cookie).containsAttribute(ClientCookie.DOMAIN_ATTR)) {
    return domainMatch(domain, host);
  }
}
return false;

And the domain attribute is never set in the constructors of Cookie. If it is set, however, domainMatch does properly handle subdomains.

Example code:

  @Test
  public void testSameDomain() throws IOException {
    try (final WebClient webClient = new WebClient()) {
      Cookie cookie = new Cookie("example.com", "name", "value");

      webClient.getCookieManager().setCookiesEnabled(true);
      webClient.getCookieManager().addCookie(cookie);

      final HtmlPage subDomainRequest = webClient.getPage("http://example.com");
      System.out.println(subDomainRequest);
    }
  }

  @Test
  public void testSubdomain() throws IOException {
    try (final WebClient webClient = new WebClient()) {
      Cookie cookie = new Cookie("example.com", "name", "value");

      webClient.getCookieManager().setCookiesEnabled(true);
      webClient.getCookieManager().addCookie(cookie);

      final HtmlPage subDomainRequest = webClient.getPage("http://www.example.com");
      System.out.println(subDomainRequest);
    }
  }

Enabling DEBUG-level logging of org.apache.http.client.protocol.RequestAddCookies and running these tests results in:

• testSameDomain:

  16:44:31.815 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: mine 16:44:31.824 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: name][value: value][domain: example.com][path: null][expiry: null] match [example.com:80/] HtmlPage(http://example.com/)@1360541835

• testSubdomain (cookie not sent):

  16:44:32.420 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: mine HtmlPage(http://www.example.com/)@841166421

---

Possible workaround: Directly instantiate a BasicClientCookie (outside of HtmlUnit’s domain) and pass it to Cookie

  @Test
  public void testSameDomainWithClientCookie() throws IOException {
    try (final WebClient webClient = new WebClient()) {
      BasicClientCookie clientCookie = new BasicClientCookie("name", "value");
      clientCookie.setDomain("example.com");
      clientCookie.setAttribute(ClientCookie.DOMAIN_ATTR, "example.com");
      Cookie cookie = new Cookie(clientCookie);

      webClient.getCookieManager().setCookiesEnabled(true);
      webClient.getCookieManager().addCookie(cookie);

      final HtmlPage subDomainRequest = webClient.getPage("http://example.com");
      System.out.println(subDomainRequest);
    }
  }

  @Test
  public void testSubdomainWithClientCookie() throws IOException {
    try (final WebClient webClient = new WebClient()) {
      BasicClientCookie clientCookie = new BasicClientCookie("name", "value");
      clientCookie.setDomain("example.com");
      clientCookie.setAttribute(ClientCookie.DOMAIN_ATTR, "example.com");
      Cookie cookie = new Cookie(clientCookie);

      webClient.getCookieManager().setCookiesEnabled(true);
      webClient.getCookieManager().addCookie(cookie);

      final HtmlPage subDomainRequest = webClient.getPage("http://www.example.com");
      System.out.println(subDomainRequest);
    }
  }

Running these with the same log config results in:

• testSameDomainWithClientCookie:

  18:06:55.032 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: mine 18:06:55.032 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: name][value: value][domain: example.com][path: null][expiry: null] match [example.com:80/] HtmlPage(http://example.com/)@841166421

• testSubdomainWithClientCookie:

  18:06:55.808 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: mine 18:06:55.809 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: name][value: value][domain: example.com][path: null][expiry: null] match [www.example.com:80/] HtmlPage(http://www.example.com/)@467632528

The workaround is dirty and hackish. A cleaner solution might be to modify constructors of Cookie to set the domain attribute, or to alter HtmlUnitDomainHandler to handle subdomains.

[rbri]

Again thanks for the report, a new release is on the way.

[rbri]

Many thanks for this detailed report. We maybe had some similar findings during the last months. But so far I was not able to reproduce them because the pointer to the manual construction of the cookies was missing. Will enhance the test suite and fix it soon. You can have a look at twitter for news. Thanks again