I'm glad to hear you got to explore Hong Kong briefly. I think deciding whether to go into the city or not would be a bit of a dilemma for me at the HKG airport on a first visit layover as well but it sounds like it turned out perfectly for you.
I've had the good fortune to visit Hong Kong a couple times and found it exciting on both occasions. There are so many people so the city really feels alive. Of course all the lights and skyscrapers help with the atmosphere too.
If you find yourself there again with more time you might like to go to the peak. It's a great view and taking the Peak Tram is nice if it's not too crowded but it can be cheaper and/or faster to take the bus or Uber.
One of my other favorite experiences there was eating beef brisket soup and rice or dan dan noodles at Sister Wah. Tiny little place but fast, delicious and a good value.
Hey thanks for the suggestions! If I do get a chance to go back I'd love to get up to the top of the hill - I had been considering trying to get up there on foot but the climb was making me perspire so much I think I was upsetting the locals :) If i'd have known there was a tram things would have been different!