You are right, the community is what matters here. If you think about your posts, usually approx same people vote for your content. Well, there are some new people engaged now and then and sometimes the regular people skip some of your articles. In general, however, kind of same people vote for the people in their community. Right? What makes you successful is that there are voters that think you matter and support you and you support them in return. You absolutely can't change your circles in your every post, there are more or less the same people voting for you and then you get some new users. And that's why community matters the most. You can rely on the people you are surrounded by (surely it means that the quality of the content remains good) and they can rely on you.