Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

1
Throttling a data producer at the service level
Post Body

Say we have a "vote" service which handles upvotes and downvotes for a post or comment. We are producing a "total votes" datum that is consumed by the "page" and "comment" services. This datum does not need to be consistent - it can be updated periodically. If a post or comment is receiving a very high volume of votes, we would only want to produce the "total votes" datum periodically. However if it is a low volume, it is fine to send the "total votes" datum at the rate which it is created.

My question is - how would you implement this throttling mechanism? I think the ideal would be to implement it at the service level, before anything is even put on the messaging system. So would you just check the timestamp of the most recent vote for a particular page/comment? There is the edge case where the datum gets throttled during high volume, and then voting suddenly stops - how would the service know it needs to produce that value? I wouldn't want to do some kind of periodic database check either.

Maybe throttling at the message queue level is preferable.... I'm not super experienced with message queue performance and I think letting the message queue handle it would probably be the "least complex" situation. The message queue could just drop all messages that haven't been processed in the queue, besides the most recent one.

Thanks for your help :D

Author
Account Strength
100%
Account Age
12 years
Verified Email
Yes
Verified Flair
No
Total Karma
42,455
Link Karma
2,070
Comment Karma
40,312
Profile updated: 3 days ago
Posts updated: 1 year ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
4 years ago