For example, we could start selling mineral rights to asteroids.

Here's how I think it could work well.

Rights for any body are always capped; for example, miners can legally extract 50% from any body without regard to ownership. This is so that the sale of rights doesn't backfire and prevent investment in asteroid mining.

Mining rights are of course transferable and divisible. Owning 1% of the iron mining rights to 16 Psyche for example would mean that after 50% of the iron is extracted from 16 Psyche, 1% of the remaining would legally be yours.

How would that be handled? Well, simply converting it to sale value probably wouldn't work. The mining company could sell it to a shell company for cheap and claim that was the full value. Basically I think it would have to mean that you have the right to set the sell price for that extracted resource. And since this would be cumbersome, in practice it would just mean that the mining company would buy the rights from you before engaging in mining.

But that leads to another issue; what about those who don't want to sell? Would that put a stop to further mining?

In that case, I think the mining company should physically leave some of the extracted material on the body. But for the most part, I think most people wouldn't buy mining rights directly, but rather buy shares of hedge funds that contain mining rights. That way the sale to miners would still be likely to occur after negotiation.

Feel free to poke holes in this, I'm sure there are many and it'll be fun to think about how to address them.

What other financial instruments could work for other types of space development? Also, would you invest in them?

I flagged this as "hard science" because I hope that people will take the financial issue seriously.