The reason most people go into small business is to support themselves.
If you want to get into politics or be a social justice warrior, using your livelihood as the lightning rod, you should "expect" to have both the positive and negative outcomes.
Why would you piss-off half or more of your customers, by being so rigid. ?