The thing with HN and other forums which utilize an upvote/downvote feature, is that it will always devolve into using downvote as a way to disagree with a post, or to discourage certain opinions, then coming up with justification for the downvote only after being called out on it.

I've accepted this inevitability, but that doesn't mean my opinion on issues will change, the best you can do is to ignore your karma when it comes to controversial topics or don't bother posting on them at all.