Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

2
scraping data, getting invalid syntax
Post Body

Hey guys, sorry in advance if this question is a little obvious, I'm new to programming and Python. I am trying to scrape data off a website with over 30,000 products. I followed a YouTube video that broke it down and was able to successfully print 250 item names in the terminal.

Once I started adding the variants and imagesrc code and pandas to convert the data to a CSV file, it gives me an error of invalid syntax.

I was also wondering if there is anyway to download all 30,000 products in one go or would I have to modify '&page=1' to '&page=2', '&page=3' and so on.

I pretty much just need the product name, tags [1] and [3], image(s) if there is one and price. Looking for some guidance on how to do this.

I have attached the code below that is giving me the invalid syntax error. Here is a screenshot of how it looks as well.

https://ibb.co/QMv0nQ1

import requestsimport jsonimport pandas as pdurl = 'https://plus.mvrwholesale.com/products.json?limit=250&page=1'r = requests.get(url)data = r.json()product_list = []for item in data ['products']:title = item ['title']handle = item['handle']product_type = item['product_type']for variant in item['variants']:price = variant ['price']sku = variant['sku']available = variant ['available']product = {'title': title,'handle': handle,'created': created,'product_type': product_type,'price' : price,'sku': sku,'available' : available,

}product_list.append(product)

df = pd.DataFrame (product_list)df.to_csv('testrun.csv')print('saved to file.')

Author
Account Strength
50%
Account Age
2 years
Verified Email
Yes
Verified Flair
No
Total Karma
n/a
Link Karma
3
Comment Karma
n/a
Profile updated: 3 days ago
Posts updated: 2 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
2 years ago