Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Splash timeout on all requests.

See original GitHub issue

I have set up docker to work with splash. Docker starts up fine but even the simple pages like google.com time out. Follows the splash terminal output for splash startup.

docker run -p 8050:8050 scrapinghub/splash

2016-02-26 03:39:07+0000 [-] Log opened.
2016-02-26 03:39:07.903064 [-] Splash version: 2.0.1
2016-02-26 03:39:07.903849 [-] Qt 5.5.1, PyQt 5.5.1, WebKit 538.1, sip 4.17, Twisted 15.5.0, Lua 5.2
2016-02-26 03:39:07.903997 [-] Python 3.4.3 (default, Oct 14 2015, 20:28:29) [GCC 4.8.4]
2016-02-26 03:39:07.904119 [-] Open files limit: 524288
2016-02-26 03:39:07.904248 [-] Open files limit increased from 524288 to 1048576
2016-02-26 03:39:08.112195 [-] Xvfb is started: ['Xvfb', ':1047', '-screen', '0', '1024x768x24']
2016-02-26 03:39:08.216409 [-] proxy profiles support is enabled, proxy profiles path: /etc/splash/proxy-profiles
2016-02-26 03:39:08.364839 [-] verbosity=1
2016-02-26 03:39:08.365024 [-] slots=50
2016-02-26 03:39:08.365640 [-] Web UI: enabled, Lua: enabled (sandbox: enabled)
2016-02-26 03:39:08.368755 [-] Site starting on 8050
2016-02-26 03:39:08.369102 [-] Starting factory <twisted.web.server.Site object at 0x7fb0ee1c4860>

Can you please help me with this.

Issue Analytics

State:
Created 8 years ago
Comments:10 (2 by maintainers)

Top GitHub Comments

1reaction

ahefnawicommented, Dec 19, 2018

@gotounix In order to set the timeout for each SplashRequest you have to pass it to the args parameter, for example to set to maximum limit: splash_args = {'timeout': 3600, 'wait': 0.5} Then use that in your SplashRquest like this: SplashRequest(..., args=splash_args)

1reaction

FreeTymeKiyancommented, Sep 9, 2018

@mgcuesta --max-timeout is only an upper bound of the timeout you can set per request. I didn’t find a global config to set timeout for all requests though. If you want longer timeout you’ll have to set them in each SplashRequest.

Top Results From Across the Web

FAQ — Splash 3.5 documentation - Read the Docs

HTTP 504 error means a request to Splash took more than timeout seconds to complete (30s by default) - Splash aborts script execution...

How to set splash timeout in scrapy-splash? - python

I use scrapy-splash package and set the timeout in args parameter of SplashRequest like this: yield scrapy_splash.SplashRequest( url ...

Scrapy and Splash times out for a specific site.

I've updated your Splash instance max-timeout setting. The timeout is needed for Splash to have enough time to fetch and render the website's...

scrapy-splash

For example, meta['splash'] allows to create a middleware which enables Splash for all outgoing requests by default. SplashRequest is a convenient utility to ......

Splash HTTP API — Splash 3.2 documentation

Most versatile endpoints that provide all Splash features are execute and run; ... See also: splash:on_request and its request:set_timeout(timeout) method; ...