How to get request/response meta info when errback is called?
See original GitHub issueI wonder if it’s possible to get the meta
and response.url
when errback
is called. Here is my code:
class CdDvdSpider(scrapy.Spider):
...
...
def make_requests_from_url(self, url):
return Request(url, dont_filter=True, meta={'foo': 'foo'},
callback=self.parse, errback=self.error_handler)
def error_handler(self, response): # FIXME: argument is not response
# print response.url
# print response.meta['foo']
pass
What I got is:
exceptions.AttributeError: Failure instance has no attribute 'meta'
I found a related post here on SO. It seems the latest Scrapy (0.24.4 as of this post) still doesn’t provide relevant API. If this is true, is it possible to add this feature?
Issue Analytics
- State:
- Created 9 years ago
- Comments:8 (2 by maintainers)
Top Results From Across the Web
Requests and Responses — Scrapy 2.7.1 documentation
Note that if exceptions are raised during processing, errback is called instead. method (str) – the HTTP method of this request.
Read more >scrapy - Trigger errback when process_exception() is called in ...
I didn't find it in the docs, but looking at the source I find DownloaderMiddleware.process_exception() can return twisted.python.failure.
Read more >Requests and Responses - Scrapy documentation
Note that if exceptions are raised during processing, errback is called instead. method (string) – the HTTP method of this request. Defaults to...
Read more >Scrapy - Requests and Responses - Tutorialspoint
Request(url[, callback, method = 'GET', headers, body, cookies, meta, ... The errback is a callable function to be called when an exception while...
Read more >Scrapy学习-10-Request&Response对象- 前路~ - 博客园
__name__) assert callback or not errback, "Cannot use errback ... Defaults to 'GET'. meta (dict) – the initial values for the Request.meta ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
in 1.5.1 you can get meta in this way
this is not in doc
The solution is use
response.request.meta