What's wrong with being a tourist destination? I live 1 1/2 hours south of Chattanooga in metro Atlanta, and love going to Chattanooga. I like the downtown area there MUCH more than going to downtown Atlanta!
I could see if you lived in a "mega tourist" city like Orlando how it may become irritating - with literally millions of tourists on the roads and crawling all over everything everyday. But Chattanooga isn't that kind of place. It has a handful of attractions that bring in "just enough" tourists to make the place feel alive and vibrant, without being insane.
Look at the alternative: I'm originally from Charleston, West Virginia. An errily alternate reality verson of Chattanooga... same size, similar looking downtown, sits on a river in a valley, etc etc. However, there's a difference - it's DEAD. Go downtown on any given weekend and you see just about NO ONE there. Likewise after work and the workers clear out of downtown on a weekday - no one. Gone. Ghost town. Charleston doesn't benefit from being only 2 hours from Atlanta, Nashville, Knoxville, and other areas like Chattanooga, so it doesn't get the "day trip" people from those areas who drive there for the day. Hence, they have no aquarium, no imax, no arts district, and only a handful of half empty restuarants downtown.
Then look at where I live now - Atlanta. Busy downtown. Busy Aquarium and hotel area. 5+ million people in the metro area. But in order to GET to those destinations, you have to go through the most insane drivers and traffic in all the southeast, then get downtown and step over a few hundred vagrants and bums while going from one of the few tourist attractions to another. It's growth at it's worst.
Now, would folks prefer THAT? Dead or out of control? ... or what Chattanooga is?

I'd gladly put up with Japanese tourists taking my picture over being the only person walking on a dead street, or, being surrounded by people asking me for money anyday.