From my horse farm in Boones Mill, VA to Lancaster County I'm about 5 1/2 hours. We're about 1 1/4 hours to the Virginia Horse Center in Lexington. So, Lexington and Rockbridge County are probably about 4 1/4 hours to Lancaster County.
The issue is that things are changing here also. Even Abingdon is changing. Yes, you have the quaint historic downtown but 5 minutes out of every quaint historic downtown in Virginia there is substantial development and growth. Business ain't what it used to be; quaint historic downtowns rely on that growth to survive. Otherwise they wither into dead husks.
I think you might enjoy the best piece of mind if you find a town with outlying areas that are developing sensibly and in a sustainable way. In this way the growth and development of the region won't get on your nerves. There is very little undiscovered country in Virginia. The remote and largely uninhabited counties remain remote and uninhabited for obvious reasons
Sean