Everything rhenning said is spot-on: you want to avoid Midtown Kingston, but downtown (by "the Strand") or uptown ("The Stockade District") could be good options for you. Both feel pretty urban, are pedestrian-oriented, and have a good amount of eateries, cafes, etc.
Uptown Kingston:
Kingston Stockade District - Wikipedia, the free encyclopedia
Downtown by the Rondout ("The Strand"):
Rondout, New York - Wikipedia, the free encyclopedia
It sounds like you're not too interested in rural or small-town living, but if you were, I would recommend Saugerties and Woodstock as good commutable options. (Saugerties, in particular, has a great, concentrated downtown with lots of shops and restaurants, so it may be worth a look.)
There are plenty of other fine areas in and around Kingston -- Lake Katrine, East Kingston, Port Ewen, Hurley, etc. -- so let me know if you have any other questions!