I struggle with this daily. I would never hang out with some of my coworkers if we didn't work together - their politics make me feel alienated and less-than, and they can be really insensitive about some topics. For example, at a work dinner, they talked about how stupid it was that UC Berkeley took down some recorded lectures because someone asked them to add subtitles to comply with the ADA.
I don't feel like I can confront them about it, because I'm worried that anything I say will be seen as an overreaction, or irrational, and I don't want to make it a big thing. But it's really hard to go to work and know that the people you work with think that you're stupid for supporting X or Y. I try keep my relationships with those people strictly professional, and to spend more time with the coworkers I agree with more, but it's still hard. I don't go to most after-hours events with my team anymore, and I feel like I'm missing out. :(