This post has been de-listed
It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.
1
Scrapy - limit of webpages?
Post Body
I am trying to scrapy to scrap multiple webpages, but somehow it always stops after 200 websites. Is there a way to get around that? How should I fix that?
Thanks!
Author
Account Strength
40%
Account Age
7 months
Verified Email
Yes
Verified Flair
No
Total Karma
361
Link Karma
131
Comment Karma
230
Profile updated: 22 hours ago
Posts updated: 5 days ago
Subreddit
Post Details
We try to extract some basic information from the post title. This is not
always successful or accurate, please use your best judgement and compare
these values to the post title and body for confirmation.
- Posted
- 5 days ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/scrapy/comm...