Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

StackOverflowError on WebClient.getPage

See original GitHub issue

URL: https://sustainability.asu.edu/person/angela-gonzales version: 2.39.1 VM arguments: -Xmx4096m

The following code throws java.lang.StackOverflowError after more than 10 minutes .

WebClient client = null;
try
{
  client = new WebClient();
  client.getOptions().setCssEnabled(false);
  client.getOptions().setJavaScriptEnabled(false);
  client.getOptions().setTimeout(20000);

  Page page = client.getPage("https://sustainability.asu.edu/person/angela-gonzales");
  if (page != null && page.isHtmlPage())
  {
    HtmlPage htmlPage = (HtmlPage) page;
    System.out.println("html page downloaded");
    System.out.println(htmlPage.asText());
  }
}
catch (Throwable e)
{
  e.printStackTrace();
}
finally
{
  if (client != null)
  {
    client.close();
  }
}

java.lang.StackOverflowError at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) … at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.FrameWindow.getTopWindow(FrameWindow.java:77) at com.gargoylesoftware.htmlunit.html.HtmlPage.getBaseURL(HtmlPage.java:2316) at com.gargoylesoftware.htmlunit.html.HtmlPage.getFullyQualifiedUrl(HtmlPage.java:699) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:182) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:124) at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1919) at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:251) at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:561) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:420) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:337) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:198) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:124) at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1919) at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:251) … at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:561) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:420) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:337) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:198) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:124) at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1919) at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:251) at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:561) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:420) at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:337) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:198) at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:124) at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1919) `

Issue Analytics

State:
Created 3 years ago
Comments:9 (6 by maintainers)

Top GitHub Comments

2reactions

rbricommented, Apr 17, 2020

Ok, regarding the reason - you have disabled js and the page contains a noscript tag that contains a iframe like this

<noscript>
    <iframe src=" ?    cn=ot&amp;...."style="display:none;width:0px;height:0px"></iframe>
</noscript>

In fact this results in a call to the same url as the page itself but with a different query string. And this returns the page including a noscript tag… and so on.

BTW this looks like one of the spy snippets like to be informed about the peoples not having js enabled also.

The trick in this case is the response for the iframe includes an X-Frame-Options: DENY header and this was ignored by HtmlUnit so far.

Have done some implementation and test for this. At the moment the ci build is running and if there are ne new errors i will make a new snapshot available soon (check twitter).

1reaction

rbricommented, Apr 23, 2020

Yes another funny case. Have written some test and will try to fix it.

Top Results From Across the Web

using WebClient and getpage - java - Stack Overflow

I'm trying to get information from a Html page using WebClient but I don't manage to get the ...

NullPointerException on htmlunit.WebClient.getPage #310

closeClinet - this typo indicates your stack trace is from a version before i made the fix for this. Pleae make sure you...

org.apache.cxf.jaxrs.client.WebClient.getPage java ... - Tabnine

WebClient client = new WebClient(); HtmlPage page = client.getPage("http://stackoverflow.com"); // create a submit button - it doesn't work with 'input' ...

"java.lang.VerifyError: Cannot inherit from final class" when ...

koma via StackOverflow ... loadWebResponseFromWebConnection(WebClient.java:1281) at ... getPage(WebClient.java:307) at com.gargoylesoftware.htmlunit.

'[HtmlUnit] [ htmlunit-Bugs-3003087 ] StackOverflowError when css ...

setThrowExceptionOnScriptError(false); SgmlPage page = (SgmlPage) \ webClient.getPage("http://www.cegov.cn/2010/0513/11488.html"); \ ((HtmlElement)page.