I've had this happen many times and honestly it doesn't bother me one bit.
I used to live in Florida, and work in Kissimmee, outside of Orlando. VERY heavily populated with Hispanic/Latinos, mostly Puerto Rican. On my lunch break I'd sometimes stop by a nearby Target and literally was at times the only non-Hispanic Caucasian in the store. Kind of amused me a little...but still never, ever bothered me.
The only thing I'll say...and I hope this doesn't come out badly, but sometimes the volume of what is being said in Spanish is just a little loud for my taste.

I speak softly, and I like my quiet.
IMHO, everyone is free to speak in whatever language they wish amongst themselves. As long and they're willing to attempt to communicate with me in English, even if they're language mastery is poor, I am fine with that. The funniest thing though, is that I usually understand a good deal of what is being said...and, as a redhead, I know when I hear
pelirroja, my ears perk up.
