This post has been de-listed
It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.
There were a few problems with the December mapping (specifically, Reddit Submission ids are now larger than the largest possible int value in the ES mapping). This meant we were missing a lot of December comments over the past day or two.
I have fixed that mapping issue (int -> long) and I am reloading all of December comments. This should be completed in about two hours.
Also, I'm going through the fields like subreddit_id, link_id, etc. and making sure they are base36 ids like the old API and not ints. This should be completed tonight as well.
We're going through the bug reports many of you have graciously provided and will be fixing a bunch of them over the next day.
Again, thank you all for your help and patience. The end result from all of this will be a much more robust and stable API with higher rate limits for everyone (probably 2-5 per second based on load). The new hardware can handle a lot more than the older hardware could.
I will keep you all updated but this will probably be my last post for this evening.
Some significant bugs seem to have been introduced during the migration; most notably, it no longer appears to be possible to exclude multiple authors (and, as another commenter pointed out, the author names themselves are not being properly matched either). Both of these completely break our analytics in a way that doesn't seem to be practical to work-around (we'd need to retrieve hundreds of extra pages in some instances). For example, author=!AutoModerator,!SomeOtherBot
would previously exclude both those accounts, but now it doesn't exclude either of them. If I'm reading the metadata correctly, this is because it's matching "any" of these conditions, which of course doesn't make sense when trying to exclude things.
Additionally, are the unique
, before_id
/after_id
, and distinguished
parameters functional, are there examples of how these are supposed to be used? They have never worked for me at all even before the migration, though it is possible I am just using them wrong (or even that the documentation is wrong or unclear).
Finally, is metadata=false
not the correct way to turn off metadata? It seems to be on by default now, and it seems wasteful to be returning this in cases we aren't going to be using it.
Edited to add: It seems the url
parameter does not work anymore either.
Subreddit
Post Details
- Posted
- 1 year ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/pushshift/c...