Cold Spring
Don't know if you've found what you're looking for by now, but Cold Spring is great. It's a perfect, compact small town - right by the water, as you know - that is very active, especially on weekends and in the summer. There's lots to do and see, and if you appreciate a quieter (but not boring) pace of life, then Cold Spring is for you.
In terms of other towns in the area that have both easy access to the train and river views, I'd suggest Hastings-on-Hudson and Croton. I've heard great things about Croton, and while I've never been to Hastings, I've heard similar things about it as well.
I'm not overly familiar with Long Island itself, but I can suggest Long Beach and Oceanside, both in Nassau. There are LIRR stations in both those towns and the commute to the city is reasonable. Plus, they are right on the ocean, and also have easy access to Jones beach and all it has to offer.
If you wanted to look in Rockland, Nyack is right on the water and is charming. However, the commute to the city would be a little convoluted and would involve driving to a nearby Jersey transit station or driving to the nearest MetroNorth station, which is across the TappanZee bridge in Tarrytown. It all depends on what you want though.
|