What's stopping a generational ship from turning around?

Same thing that keeps people from capsizing their entire society - since that's really what this comes down to - in other circumstances. Routine, a decently stable social structure, an appreciable amount of comfort.

When faced with the prospect of spending your entire life living on insect protein and drinking recycled bathwater

Outside of an incredibly hastily thrown-together exile fleet you'd want to, y'know. Take steps towards not doing that. And if you did that, you'd take steps to normalize those circumstances from the outset. Pre-launch even.

A very simple safeguard would simply be to have all your prospective passengers live in conditions mirrored after those on the ship for 5 years. You can leave at any point in the process and new people can join, but if you leave once you're barred for life. Boom, you've selected for people that consider all of this perfectly normal and will pass those values onto their descendants.

Assuming the generational ship is a colony vessel, how do you keep the crew on mission for such an extended period?

Engineered culture. Take inspiration from everything from military & civilian academy ceremonies to highly formalized & ritualized religions to the rather silly but seemingly effective little supermarket dances common in some places. The biggest motivator for doing things is identity. If you make being a colonist part of the core identity of the crew and showcase how everyone is signed onto it you'll go a LONG time with the majority being onboard, no pun intended.

The first generation will see it as a silly exercise. The second will consider it a set of weird habits. Later ones will understand it as being as integral to what they are as eating food.