I think I can understand that fear - but I like being in control. I actually follow people I disagree with, by conscious choice. It adds to the discussion. It keeps things fresh and interesting. I could cocoon away with some people that make me feel good about myself. But instead I keep a group of people in my feed that challenge the way I think and disbelieve the things I believe in. If I miss an intelligent post made by some one I ignore because they usually post pictures of unicorns, cats and narwhals, well, then I miss out. But I also miss all of the noise. (not that there's anything wrong with unicorns, cats, and narwhals).
it's up to us, right? we get to control the flow. I've misunderstood people, and I've been misunderstood. I had to control myself and hold back the inner flame-troll and just keep it civil. And in the end, I still don't agree with some people... but I follow a few of them just to keep things interesting.