I've been running to this issue endlessly... I have a table of data, for which there's one column that we want to sort upon in some way, but the data needs to be populated. So the entire table query needs to be slowed down as i cant use indexing to sort or skip/limit, and now need to load a ton of data into the webapp to do so.

I've seen an open source solution to this floating around, and also I've been thinking about it for a while. But has anybody used change streams to sync data from one collection to another? adding "virtual" fields that you can index/sort/query with on one single collection which are populated via the CS.

Additionally, im curious if there are other ways to go about this, one interesting feature mongo released is On-Demand materialized views. Though the major downside to them is that they're snapshotted, and I'm looking for as close to real-time as possible.

Note:

I'm working at a startup thats going through some fast growth. We started with mongo, and ended up with a bunch of use cases involving joining large sets of data. though it may be "better" to have used a relational DB, hindsight is 20/20 haha.