Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
response.body contains no HTML
See original GitHub issueMy target site issues a redirect which when I stop forcefully causes the response object to come back blank without any HTML. Also, on another site, it contains the flash screen (used to denote Loading of results).
Is there a way to simulate the page loading and then pull the HTML into the response object?
Issue Analytics
- State:
- Created 8 years ago
- Comments:6 (5 by maintainers)
Top Results From Across the Web
Response.body - Web APIs - MDN Web Docs
The body read-only property of the Response interface is a ReadableStream of the body contents.
Read more >post with no response body, which response code is better ...
You should return a 204 status code with an empty body. Our request is a POST (which obviously creates a resource). The request...
Read more >Response - Grafana k6
Name Type Description
Response.remote_ip string The IP address of the server handling the request.
Response.remote_port number The port that was connected to on the server...
Read more >Response - Postman Documentation
Response holds data related to the request body. By default, it provides a nice wrapper for url-encoded, form-data, and raw types of request...
Read more >API Reference - Express 4.x
A new body string containing the parsed data is populated on the request object after the middleware (i.e. req.body ), or an empty...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
You are trying to get a javscript-only page. At least there is really nothing much to see on that link without javascript. Disable it to see what scrapy sees. The loaded page you expect requires a DOM and javascript execution to be built/rendered.
In that case, you may use any of these: Splash + ScrapyJS or phantomjs / SlimerJS or Selenium with webdrivers, either standalone or as middlewares in scrapy. There may be more alternatives that I am not aware of. Take your pick.
A very useful guidance, thank you~