We both know it happens, even if in the morning for both of us it almost feels like it was a dream

We know he does this practically asleep, but he's a light sleeper so if I talk to him he responds and we've managed to discuss this both right when it has happened, and afterwards in the morning

He feels bad about it and we've agreed that I should take his hand and push him away when it happens, it solves the issue for the night, but it happens almost every night

It is pretty annoying both because I don't want to be woke up, and because he's essentially touching me without my consent, and no, I don't like CNC and is not part of out dynamic, but we don't know what else to do

So, if anybody has gone through this, or has heard of it, is there any permanent or more long term solution?