Issue with running scrapy spider from script.
See original GitHub issueHi ! I am trying to run this code
import csv
import scrapy
class SouqSpider(scrapy.Spider):
name = "Souq" # Name of the Spider, required value
start_urls = ["http://deals.souq.com/ae-en/"] # The starting url, Scrapy will request this URL in parse
# Entry point for the spider
def parse(self, response):
for href in response.css('.sp-cms_unit--ttlattr(href)'):
url = href.extract()
yield scrapy.Request(url, callback=self.parse_item)
# Method for parsing a product page
def parse_item(self, response):
original_price = -1
savings=0
discounted = False
seller_rating = response.css('.vip-product-infoats .inline-block small::text'). extract()[0]
seller_rating = int(filter(unicode.isdigit,seller_rating))
# Not all deals are discounted
if response.css('.vip-product-infobhead::text').extract():
original_price = response.css('.vip-product-infobhead::text').extract()[0].replace("AED", "")
discounted = True
savings = response.css('.vip-product-infossage .noWrap::text').extract()[0].replace("AED", "")
yield {
'Title': response.css('.product-title:text').extract()[0],
'Category': response.css('.product-title span a+ a::text').extract()[0],
'OriginalPrice': original_price,
'CurrentPrice': response.css('.vip-product-infoice::text').extract()[0].replace(u"\xa0", ""),
'Discounted': discounted,
'Savings': savings,
'SoldBy': response.css('.vip-product-infoats a::text').extract()[0],
'SellerRating': seller_rating,
'Url': response.url
}
However , i run this script i get this following error
class SouqSpider(scrapy.Spider): AttributeError: 'module' object has no attribute 'Spider'
anyone who can fix this error
Issue Analytics
- State:
- Created 6 years ago
- Comments:7 (1 by maintainers)
Top Results From Across the Web
Common Practices — Scrapy 2.7.1 documentation
By default, Scrapy runs a single spider per process when you run scrapy crawl . However, Scrapy supports running multiple spiders per process...
Read more >Issue with running scrapy spider from script. #2473 - GitHub
Hi, I'm trying to run scrapy from a script like this: import scrapy from scrapy.crawler import CrawlerProcess class MySpider(scrapy.Spider): ...
Read more >scrapy run spider from script - python - Stack Overflow
You can just create a normal Python script, and then use Scrapy's command line option runspider , that allows you to run a...
Read more >How to Run a Scrapy Spider from a Python Script - Finxter
In the normal scrapy workflow, we begin by starting a project with scrapy's startproject command.
Read more >How to Run Scrapy as a Stand-Alone Script | Teracrawler
While Scrapy is super useful, sometimes it could be a little stifling to create a project and then a spider and all the...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Have you checked the version of scrapy that you are using ? It is possible that you are using an older version of scrapy and referring the new documentation. If it is an older version of scrapy, then run the following command and check again.
pip install scrapy --upgrade
still facing same error @ppm-khusbu