Has anyone here lowered their wee storm? How badly did it affect handling?

I have a 2014 that I’ve put about 50k on. I go trail riding with my friend who owns a wr250. I really enjoy some of the technical single track stuff, and the bike takes everything I throw at it. I also enjoy the sideways looks I get dragging a big heavy bike up a rocky hill. I’m 5’9” with a 30” inseam. I always pick which foot I’m going to put down when I come to a stop light. As I get older, I find myself getting more cautious. I don’t want to stop riding these trails, but my bike is heavy and I’m feeling the mental limitations holding me back. I’ve always been against altering the geometry of a bike, but I find myself considering it. I do a lot of touring, so a wr250 doesn’t suit 90% of my riding.

Thoughts?