Taxes are indeed lower in VA than in DC or MD, but they still aren't necessarily low. Depends how things are where you are now. Car insurance is also lower in VA, but that might not matter much if you won't have a car. It is much more difficult to get by without a car in VA than it is in DC. Things in NoVa were laid out (almost all of it since WWII) with cars in mind. The rents for a place that is in walking distance of a Metro station will also be at a premium. Of course, that depends on what you consider walking distance. The traditional advice would be to recommend the Ballston Corridor, which is pretty much everything between the Court House and Ballston stations along the Metro Orange Line. That is mostly high-density, smart-growth development with all the amenities and nightlife built right in. Clarendon is sort of the hub. Yuppie heaven, but it can be expensive. Crystal City is a smaller alternative comprised mostly of high-rise buildings and with not quite the same level of amenities and nightife. Lots of young people live there too, and it's not quite as expensive. The not so traditional advice would be to look around further out. There are a lot of rental units that are within walking distance of both the Dunn Loring and Vienna Metro stations, and the rents, while still not low, are a little more affordable still. There would be significant challenges to living in those areas without a car, though.
If you haven't come across it yet, here is the
map of the Metrorail system superimposed on a standard street map. You can learn a lot about where things are and how you can get from one place to another by using that.