Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Start Scrapyd with the `daemon` command

See original GitHub issue

I would like to start scrapyd as service but when I start scrapyd, if I close the SSH session the service scrapyd close automatically.

When I try to start as service like this I have an error :

root@vps:~# service scrapyd start
scrapyd: Failed to start scrapyd.service: Unit scrapyd.service failed to load: No such file or directory.

And when i try to start daemon scrapyd the CURL request return :

{"status": "error", "message": "Use \"scrapy\" to see available commands", "node_name": "vps"}

Can someone help me to start my scrapyd as service please !

Issue Analytics

State:
Created 8 years ago
Comments:14 (6 by maintainers)

Top GitHub Comments

1reaction

ricoxorcommented, Mar 10, 2016

Thank’s Digenis I find the solution : daemon --chdir=/home/Crawler scrapyd

I need to set the --chdir to load the service on the Scrapy folder !

0reactions

Digeniscommented, Mar 4, 2017

twistd can also daemonize scrapyd it’s just that the default console script doesn’t. It also need some additional arguments for the working directory, path to pid file and logging.

For systemd, see twisted’s latest doc: http://twistedmatrix.com/documents/current/core/howto/systemd.html There’s a ticket for scrapyd on systemd: #102

Top Results From Across the Web

Preferred way to run Scrapyd in the background / as a service

To have scrapyd run as daemon, you can simply do: ... This rather complicated command starts Scrapyd image in the background ( -d...

Scrapyd Documentation

Scrapyd is an application (typically run as a daemon) that listens to requests for spiders to run and spawns a process.

Scrapyd — Scrapy 2.7.1 documentation

Scrapyd ¶. Scrapyd has been moved into a separate project. Its documentation is now hosted at: https://scrapyd.readthedocs.io/en/latest/ ...

The Scrapyd Guide - Deploy & Schedule Your Scrapy Spiders

The Complete Guide To Scrapyd: Deploy, Schedule & Run Your Scrapy Spiders ... And then start the server by using the command: ......

vimagick/scrapyd - Docker Image

scrapyd · docker-compose.yml · Run it as background-daemon for scrapyd. $ docker-compose up -d scrapyd $ docker-compose logs -f scrapyd $ docker cp ......