There are several outlet centers in Georgia, but all of them are an hour or more from Atlanta proper with no viable public transportation. There are centers in Calhoun (on Interstate 75 northwest of Atlanta), Dawsonville (Georgia Hwy 400 north/northeast of Atlanta), and Commerce/Banks Crossing (Interstate 85 northeast of Atlanta). There's also a Tanger Outlet Center in Locust Grove, south of Atlanta on Interstate 75.
The only outlet mall (sort of) in the metro area proper is Discover Mills on Sugarloaf Parkway in Gwinnett County. You can get there via a combination of MARTA and Gwinnett County Transit, though the operating hours for GCT are somewhat limited (it's mainly targeted to commuters heading into the city from Gwinnett, not tourists heading out to Gwinnett from the city).
Here's a Google Map showing these locations:
Atlanta area outlet centers