The area that is close to downtown is not bad. Main St, White St, Confederate Ave, Reid St. have some beautiful old homes. A former TV weatherman and his wife bought one of the homes on Main St in the district and opened a B&B.
I was born in St. Phillips Hospital which at the time was located on the corner of N. Confederate Avenue and Pickens St. The hospital was operated by The Franciscan Sisters of Peoria and due to lack of funding had to close up in the late 1950s. A little funny here ... at one time the hospital was on one corner and their surgery room was across the street ... one had to be pushed on a gurney to surgery through the front yard and across the street with nuns stopping any traffic and one just prayed for no rain!
I would be a little leary of the Marshall St, Workman St and those areas further from the downtown area.
They have a website which I am sure you have visited:
Historic East Town
You probably could attend one of their meetings which are normally held at Park Baptist Church on Main St. to get more info.