This post has been de-listed
It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.
Hey guys.
As the title says, we recently hit the user-notes size limit in /r/AskReddit. If you're not aware, the store for user-notes is at /r/subreddit/wiki/usernotes. Here are a few things we learned:
- Wiki pages seem to max out at around 300KB. Our list was right on top of that.
- RES seems to be wrong about the max amount of characters allowed on a wiki page. If you go by that, we should have only been at 50%.
- For us, 300KB of user-notes equated to ~2900 user-notes for ~2600 users.
- You don't get much notification that you've hit that limit. Notes just kinda stop working (while potentially tricking you into thinking they are working by showing up in your browser).
We REALLY didn't just want to prune notes before a certain date or anything else heavy like that because we find that the legacy notes are the MOST useful notes because they keep us consistent in handling users when they come to our attention again. Keeping that in mind, we discussed a few options in freeing up some space. Here are some of the things we discussed:
- Pruning all shadowbanned users.
- Truncating all text notes to a certain character size.
- Remove all links in text notes.
- Re-marking users that we consider perma-banned in an efficient manner with minimal characters.
Unfortunately, all of these would require some significant scripting to work right. We still might pursue these routes (because our current fix won't last that long) but we went with something much simpler for the short term. Our current fix is: EDIT Disregard. Told this was a bad idea.
Note: We removed the timestamps, but were told this might break future updates. Please don't try it without further guidance from the /r/toolbox team.
By removing the timestamps from the notes, we were not only able to keep pretty much everything we consider valuable in the notes, but were also able to free up ~40KB of space in our notes. Now, this is only a short term fix. By our current numbers, this should allow us to get in another ~400 notes before we run into problems again. However, that should be enough time for us to try and implement one of our other above solutions. If anyone has the resources to actually build the capability to do any of those functions (especially the removal of shadowbanned users one), that would be extremely useful to lots of subreddits who are running into this issue.
I've had some discussions with the /r/toolbox team regarding how this can be fixed programmatically but I'll let them discuss any potential implementations of those solutions since it's not my place. I've also brought this up to the reddit admins for their consideration.
Anyways, I hope this is helpful for you guys. Thanks to both /u/Deimorz for his help and the /r/toolbox team for being extremely responsive and easy to work with in trying to figure out a long term solution to this.
Post Details
- Posted
- 9 years ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/toolbox/com...