If I were young and single and either worked downtown or attended the University of Louisville, I definitely would consider living downtown. It's really improving and there is a ton of stuff going on.
LouisvilleKy.gov - 2006 - Louisville's Downtown Alive with Development
When I retire, I would seriously consider moving to a condo on the river just up river road, where there are great views of the city. There is also a new, classy condo development going up near the stadium, but the name escapes me now. Gallery Square Lofts? (Edited to add: I think it's Mercantile Gallery Lofts.)
Clifton Lofts looks interesting.
Clifton Lofts Louisville Condominiums | RE/MAX Properties East | Louisville Real Estate
Like all inner city living, there are good points and bad, and you'd have to know what areas to avoid. But it is certainly worth considering.