Scraping with BeautifulSoup and style tags.

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

Post Body

I'm trying to scrape the abstracts for some NBER papers ( example). The tag surrounding the abstract is:

But I can't figure out a way to select it. My current attempt

res = requests.get(url)
res.raise_for_status()
paper_soup =  bs4.BeautifulSoup(res.text,"lxml")
    paper_abstract = paper_soup.findAll("p",{"style" : "margin-left: 40px; margin-right: 40px;text-align:       justify"})

with which I get the error

'charmap' codec can't encode character '\u201c' in position 997: character maps to <undefined>

Author

Account Strength

100%

Account Age

13 years

Verified Email

Yes

Verified Flair

Total Karma

38,661

Link Karma

3,840

Comment Karma

34,803

Profile updated: 3 days ago

Posts updated: 6 months ago

Ponderay

Subreddit

r/learnpython

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.

Posted: 7 years ago
Reddit URL: View post on reddit.com
External URL: reddit.com/r/learnpython...