Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

18
[Update] Why are monthly dumps taking so long?
Author Summary
Stuck_In_the_Matrix is in update
Post Body

Monthly dumps are taking significantly longer to process right now because of how Reddit deals with large spammers. Spammers will make millions of comments to specific subreddits in the hopes it boosts their relevancy on search engines. This causes some days to have 20% or more of all comments being nothing but spam from specific accounts.

When monthly dumps are processed, there is a second pass made to check all missing ids to make sure as much data is collected as available. The issue is that when the first pass is done, there are millions of missing ids because these spam comments get removed from Reddit at some point.

It will be necessary to change the logic of the "fill the gaps" script so that it looks at the Pushshift API to see if the ids that are missing are from spammers.

With that said, May is almost done and June / July will start soon. Submissions are a bit easier to handle and will be brought up to date over the next 2-3 weeks.

Thanks!

Author
Account Strength
100%
Account Age
11 years
Verified Email
No
Verified Flair
No
Total Karma
143,730
Link Karma
34,810
Comment Karma
108,242
Profile updated: 2 days ago
Posts updated: 6 months ago

Subreddit

Post Details

Location
We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
5 years ago