Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
I ran into a problem when I used ipython in a scrapy shell
See original GitHub issueI used this command, scrapy shell 'www.baidu.com', and then used response.body.
When I input resp and click Tab
As you can see, there was a display error
Use the command completion function is encountered a display error
Issue Analytics
- State:
- Created 6 years ago
- Reactions:4
- Comments:9 (3 by maintainers)
Top Results From Across the Web
Scrapy shell Error - python 2.7 - Stack Overflow
Your settings.py file should be in the root of your Scrapy project folder, (one level deeper than your scrapy.cfg file). If you've already...
Read more >Scrapy shell — Scrapy 2.7.1 documentation
The Scrapy shell is an interactive shell where you can try and debug your scraping code very quickly, without having to run the...
Read more >Scrapy Shell Doesn't Use Ipython - ADocLib
Solving specific problems It's meant to be used for testing data extraction code, but you can actually use it for testing any kind...
Read more >Easy web scraping with Scrapy | ScrapingBee
You can configure Scrapy Shell to use another console instead of the default Python console like IPython. You will get autocompletion and ...
Read more >Run Scrapy code from Jupyter Notebook without issues
@wait_for is used for blocking calls into Twisted Reactor thread. Click here to learn more about this. Voila! No more error. The script...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I think it is reasonable to expect that DEBUG messages are shown when log level is set to DEBUG, so I’m not sure it is an upstream issue now.
parsolibrary emits debug messages, and they have a right to do so. Scrapy configures a global “DEBUG” level, which doesn’t play well with this library, as it is used in autocompletion (?). I think we can put a workaround to Scrapy itself, increasing log level forparsologgers to INFO inscrapy shell.I have the same issue with
ptpython.