While I cannot speak to enjoying Cold Spring with children [as I do not have any], I grew up in nearby Southern Dutchess county [and still have family in the area] and, as a consequence, have taken many trips there, including a few recent ones. Though my knowledge is not extensive, nor would I try to pretend it to be, perhaps I can offer a few pointers.
First, if you are going to take the train to Cold Spring, you would ride Metro North's Hudson Line [on a northbound/to Poughkeepsie train] and get off at the Cold Spring station. From there, you will be in walking distance (about 5-10 minutes, depending on where you are headed) to the village center, ie Main Street. Alternately, you could take a taxi from the train station and be there in a couple of minutes.
Here is a link to the weekday Hudson Line timetable and
here is a link to the weekend/holiday timetable. And
here are Google Maps directions from the train station to the beginning of Main Street.
Main Street is the center of Cold Spring and a significant part of what attracts visitors to the village. It is lined with quaint, historic buildings and a variety of restaurants, antique stores and other interesting, little shops. Much of the fun of visiting Cold Spring is simply walking through Main Street, exploring the various stores. Though I have no specific recommendations, you will have no trouble finding a place to stop and get lunch or a cup of afternoon tea; there will, in fact, be many places for you to choose from.
In that same area, you are also given access to some beautiful Hudson River views, including grassy areas with benches to relax on. If I remember correctly, there are also a few restaurants, with one being very close to the actual river, with said views, as well.
If you are coming up on a weekend (or Friday) between May and December, your family may also enjoy taking a ride on the Cold Spring Trolley to
The Foundry Museum or other location, which I have heard can provide a relaxing way to see Cold Spring and the surrounding area. The one catch is that it only runs every couple of hours, so you should be confident that you will enjoy spending that time at your destination while waiting for the return trolley.
Here is a link to the trolley schedule; there is a phone number to call on that page if you have any further questions.
Good luck!