There’s a lot of reasons it can happen, but a lot of it comes down to the fact that hats are hooped so differently from other items because they can’t be laid flat. No matter how tightly you hoop the hat there will probably still be a small amount of movement. So often a design that might sew flawlessly on a flat piece will run into some tiny inconsistencies on hats.

Things that you can do to help- Hoop as tightly as possible, and clip down the sides of the hat. Increase your density for hats and add an edge run underlay to any larger text. Slow down your machine (I usually run hats at 750 max).

And yes, your hat choice can effect this as well, some fabrics just embroider cleaner than others. Looks like you might be using the Yupoong 6006? Love those hats, they have so much room to embroider and are amazing for big bold designs. But you do tend to have a bit more jagged edges on them than some others because it’s a thinner fabric. Try the same design on a YP 6089 or a FlexFit 210 and you’ll see a cleaner edge usually. Downside is those are 6 panels and you’ll have the center seam to deal with.