Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

5
We recently hit the user-note size limit in AskReddit. Here's how we've temporarily fixed it and what we learned in the process.
Post Body

Hey guys.

As the title says, we recently hit the user-notes size limit in /r/AskReddit. If you're not aware, the store for user-notes is at /r/subreddit/wiki/usernotes. Here are a few things we learned:

  • Wiki pages seem to max out at around 300KB. Our list was right on top of that.
  • RES seems to be wrong about the max amount of characters allowed on a wiki page. If you go by that, we should have only been at 50%.
  • For us, 300KB of user-notes equated to ~2900 user-notes for ~2600 users.
  • You don't get much notification that you've hit that limit. Notes just kinda stop working (while potentially tricking you into thinking they are working by showing up in your browser).

We REALLY didn't just want to prune notes before a certain date or anything else heavy like that because we find that the legacy notes are the MOST useful notes because they keep us consistent in handling users when they come to our attention again. Keeping that in mind, we discussed a few options in freeing up some space. Here are some of the things we discussed:

  • Pruning all shadowbanned users.
  • Truncating all text notes to a certain character size.
  • Remove all links in text notes.
  • Re-marking users that we consider perma-banned in an efficient manner with minimal characters.

Unfortunately, all of these would require some significant scripting to work right. We still might pursue these routes (because our current fix won't last that long) but we went with something much simpler for the short term. Our current fix is: EDIT Disregard. Told this was a bad idea.

Note: We removed the timestamps, but were told this might break future updates. Please don't try it without further guidance from the /r/toolbox team.

By removing the timestamps from the notes, we were not only able to keep pretty much everything we consider valuable in the notes, but were also able to free up ~40KB of space in our notes. Now, this is only a short term fix. By our current numbers, this should allow us to get in another ~400 notes before we run into problems again. However, that should be enough time for us to try and implement one of our other above solutions. If anyone has the resources to actually build the capability to do any of those functions (especially the removal of shadowbanned users one), that would be extremely useful to lots of subreddits who are running into this issue.

I've had some discussions with the /r/toolbox team regarding how this can be fixed programmatically but I'll let them discuss any potential implementations of those solutions since it's not my place. I've also brought this up to the reddit admins for their consideration.

Anyways, I hope this is helpful for you guys. Thanks to both /u/Deimorz for his help and the /r/toolbox team for being extremely responsive and easy to work with in trying to figure out a long term solution to this.

Author
Account Strength
100%
Account Age
17 years
Verified Email
Yes
Verified Flair
No
Total Karma
83,239
Link Karma
17,294
Comment Karma
65,617
Profile updated: 4 days ago
Posts updated: 7 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
9 years ago