Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

2
JQuery and web scraping?
Post Body

I am trying to scrape a returned set of business licenses from a municipal site that whenever you use a search term, let's say 'Fuel', it appears to be handled by a jQuery script that returns the results. The problem is, I have been unable to reproduce the search string the site sends to their database. Using the element inspector in my browser, as well as monitoring the network traffic sent to the site from my computer, doesn't reveal how the results are obtained. The results are clearly visible in my browser and in the source code, but as I using the 'requests' library, I can't accomplish the same results as I can't replicate their search command.

Is there anyway to find the actual search parameters or, if I believe what is happening is correct, which is the search term is handled by java script and jquery to return the results, am I out of luck due to the javascript handling the search feature?

Thanks and I apologize for my poor description. Although I have done many websites in the past, I have never needed to interface with a database so the backend part kind of eludes me.

Author
Account Strength
100%
Account Age
10 years
Verified Email
Yes
Verified Flair
No
Total Karma
15,217
Link Karma
4,341
Comment Karma
10,493
Profile updated: 2 days ago
Posts updated: 5 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
3 years ago