When I'm very annoyed or frustrated, or even angry, I get very quiet because I don't want to overreact or say something regrettable. I'll talk about it when I'm ready, but in the meantime, I speak softly and measured and make it a point not to be rude or short. It usually requires a good amount of self-control, especially if I am really upset.

However, this also creates problems when my partner recognizes that I'm upset because I'm quieter/less outgoing than usual. Four times out of five, it turns into a fight for "being passive aggressive and withholding affection," not to mention refusing to say what's wrong. It ends up going circular because my response is that I didn't say anything because I didn't want to fight about it yet here we are because I stayed quiet.

Do I need to learn how to compartmentalize a little better? How do keep frustration from showing while I'm working through it?

Edit #1:

I appreciate your responses. I think a big part of my "being upset" is more accurately "feeling bad for feeling bad." My animal brain gets upset at stupid shit, and I really don't want to fight about it or even talk about it. I don't want to be upset at it, but my ability to rationally let the frustration bounce off gets compromised somehow and I get stuck in a holding pattern. Talking about it will only make two people mad about something that zero people should be mad about.

Edit #2: Many of the responses seem to be equating being quiet(er) and not starting conversations with the "silent treatment" or "stonewalling." Nobody is being ignored or treated as less than, but in those moments I'm not feeling particularly warm, so I'm obviously less affectionate and less wanting to go out of my way to do things.