allow users to pass spider arguments via url

When running Scrapy from command line you can do:

> scrapy crawl foo_spider -a zipcode=10001

but this is NOT possible with ScrapyRT now. You cannot pass arguments for spiders, you can only pass arguments for request. Adding support for “command_line” arguments is not difficult to implement and seems important IMO.

You could simply pass

localhost:8050/crawl.json?spider=foo.spider&zipcode=10001&url=some_url

EDIT: clarify we’re talking about passing arguments to API via url

[aleroot]

I have tested the pull request and it is working well, I think should be merged.

[dotungvp1994]

@pianista215 : Yeah, i know i will to try @gdelfresno trick and working but i thought pass arguments to meta data is the same. if you pass arguments by curl -XPOST -d '{ "spider_name":"XXX", "start_requests":true, "request":{ "meta": {"lookup_until_date": "23-09-2017" } } }' "http://localhost:9081/crawl.json" >> response you can get loockup_until_date by response.meta["lookup_until_date"] 😄. Yeah, i have seen you made with change in your repositories, i will to try it. 👍