I think it works ok with more stake = more influence as SP distribution is spreading. It'd take time. I'd agree there should be a balance between # of votes and stake of votes though, although I'm not too sure how to resolve that. Someone could just make many mini accounts and vote in a train and abuse the # of votes..