The area blocked in by S. Duke, Proctor, and Vickers area has some old mansions that are fun to see. Hill House is one:
The Junior League of Durham and Orange Counties, NC - Headquarters Greystone Inn is another:
Greystone Webpage
See this, too, from the Preservation Society:
Preservation Durham, North Carolina - Durham's Historic Neighborhoods (http://www.preservationdurham.org/places/nhoods.html - broken link)
(Make sure if you go to the Hope Valley area to drive around the country club neighborhood--you'll see the golf course--and not just the newer developments.)