I breast fed my daughter 16 years ago. I found it bizarre that people would rather hear a screaming baby than ignore a baby breast finding quietly in a corner.
Some people are just completely nuts.
Keep doing what you are doing: Your baby will have better hearing, memory and fewer colds than other kids.