Same here. My point/argument, upvoted or downvoted, is still my point. If someone doesn't want to engage it because I have downvotes, then I don't think I want to hear them. Upvotes or downvotes shouldn't change the validity of my argument. A racist post with faulty arguments is still invalid even if it has all the upvotes and is at the front page. Meanwhile I tend to post on throwaways on reddit anyways, so I mind the downvotes even less.
Sure, but it's about visibility and about normalizing attitudes and behaviors. A post with 500 upvotes saying "OP is a faggot" and a response at -20 saying "hey, that's not cool to say" is going to make people think that it's 1. okay to call others faggots and 2. discouraged to speak up against that kind of language. Downvotes really are distributed silencing mechanisms. They reduce people's ability to see, and willingness to consider, contradictory viewpoints.