thankfully when i was visiting Krabi i had a local guide who had lived there for many years. When we went to the Hong Islands we did exactly what you are describing above and we chilled out in the morning, had a nice breakfast opposite where the boat was and our boat guy came and got us when he got the word on his mobile from other people that the tour groups were far enough on their way that they would be gone by the time we got there.

What we encountered when we arrived was a near empty island... on an island that is one of the most popular islands in the area. This is definitely very solid advice!