Does anyone else tend to alienate themselves from their work colleagues?

I actually LIKE to be and unfortunately my colleagues always take it personally. It’s not that I have anything against them - because I don’t. They are generally very nice people. I just don’t like people. If they talk to me , i speak to them. I just don’t go out of my way to participate in after work social events - hell I even try to avoid during work social events. I only engage with them when necessary (team work/projects) .

I don’t think my colleagues don’t like me , most of them do. Just sometimes they may “tease” that I’m not part of the group or part of the team because I’m “never there [in the office]” - despite me being there as little or as often as my other colleagues.

I don’t care if they are unhappy with me not being pressed to get on a personal level/family- like level with them. What I don’t like is how I sometimes get passive aggressive attitudes and snide comments because I am reserved . They make assumptions like I don’t like them or think I’m better than them (all “joking” accusations that have been made - several times) and shun me because of it. It’s just VERY high maintenance and annoying. It stresses me out because it is SO unnecessary.