I like how you said "when" not "if"

I see two possible problems:

1. People will abuse down voting because they simply disagree with what someone said, not because what was said was actually hateful.
2. There will be down voting wars, kinda like turf wars. People will take sides and once it all breaks loose, then everyone will be a casualty.