I agree with cost you that when you start to do what you desire and just ignore the opinion of others, you can end up being socially ignored. For your Question.

How do you determine when it's worth it to accept that cost?

You should analyze that how much it matters to you to live your life in a certain way. Is it possible for you that if you don't follow that certain path that is criticized by others, you will still be living a fulfilling life. I will give you an example of my own here just to clarify when we can ignore others.

I live in a conservative society where earning living from the internet is thought to be a dream that internet markets show to fool people. That's the public opinion around here in Pakistan. Not all believe this but 99.99 percent of Pakistanis believe this is true. In my social circle, everybody believes that I am living in a dream and wasting my time on the internet and often get opinion from others that I should do something useful with my time. But, I ignore every opinion my friends and family has about my career and continue to live my dream. As a result, I am kind of freak in my social circle. Not invited to the parties or gatherings. Is it ok? Yes it is. I can handle that. But what I can't handle, is to live the traditional lifestyle, wake up go to work. Come home do your chores sleep and then the same for the next day. I can risk everything to get out of this cycle. Coz I just don't want to stuck at the same place throughout my whole life. I want to travel, I don't want to live local rather I want to live an international life and that is why I ignore my critics. I choose to be socially alone. May you have get my point and the answer to your question.