scrapy FormRequest ignore Content-Type other values
See original GitHub issueThe following code
super(FormRequest, self).__init__(*args, **kwargs)
if formdata:
items = formdata.items() if isinstance(formdata, dict) else formdata
querystr = _urlencode(items, self.encoding)
if self.method == 'POST':
self.headers.setdefault('Content-Type', 'application/x-www-form-urlencoded')
self._set_body(querystr)
else:
self._set_url(self.url + ('&' if '?' in self.url else '?') + querystr)
Although the FormRequest base class Request sets the request header based on the value of the headers band, such as setting Content-Type = “application / json; charset = utf-8”, the code
self.headers.setdefault('Content-Type', 'application/x-www-form-urlencoded')
sets the Content- Type value back to "application / x -www-form-urlencoded ". He did not consider headers own Content-Type situation.
Issue Analytics
- State:
- Created 6 years ago
- Comments:5 (4 by maintainers)
Top Results From Across the Web
Requests and Responses — Scrapy 2.7.1 documentation
It accepts the same arguments as the Request class, taking preference and overriding the values of the same arguments contained in the cURL ......
Read more >Python Scrapy override content type to be multipart/form-data ...
Just use scrapy.http.FormRequest instead of scrapy.Request, passing the parameters in the formdata argument. Sample code:
Read more >Requests and Responses - Scrapy documentation
Scrapy uses Request and Response objects for crawling web sites. ... Returns a new FormRequest object with its form field values pre-populated with...
Read more >Scrapy Documentation - Read the Docs
Scrapy (/skrepa/) is an application framework for crawling web sites and extracting structured data which can be used.
Read more >Easy Way to Handle Form POST with Python Scrapy - YouTube
Handling POST requests can be a little tricky with scrapy. In this video, we will see a specific scenario, where the POST request...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hi @andywubit - I’m inclined to close this unless we can get a reproducible example case to demonstrate the issue. It appears, from my reading and others, to be perhaps a case of misunderstanding. If you think we could improve our documentation to make this clearer, that’s a potential way we could take this?
Looking forward to your response.
@andywubit can you provide a code snippet that reproduces your issue? thanks