I'm not familiar with this swim club, but we live very close to our swim club in another part of Fairfax County. If you have kids, or enjoy swimming and tennis on your own, you will love living within walking distance. I checked the club's website, and (unlike some others in Nova), it does not have a waiting list to join.
RVSTC | Rolling Valley Swim and Tennis Club, Springfield, VA If this is a typical club, the only real noise/crowded parking conditions will happen during swim meets. Most clubs have between 6-8 home meets on Saturday mornings and Monday evenings during the summer. The Saturday meets start at 9 am, but setup/warmups begin before 8 am, and sometimes the starters' horn or the PA system can be bothersome to neighbors who like to sleep in on weekends.
Otherwise, you should not have a problem. You should only expect a parking lot "thug" issue if the neighborhood is otherwise a "thug" neighborhood, which is unlikely.