Hi there,
There are plenty of options within the Hudson Valley that would fulfill your needs and are convenient to NYC and the other points you mentioned. Understand however that the cost of living will be significantly higher in New York than in Jacksonville.
For perspective (numbers via City-Data):
Mar. 2012 cost of living index in Jacksonville: 89.3
(less than average, U.S. average is 100)
Mar. 2012 cost of living index in Middletown: 137.5
(high, U.S. average is 100)
While it would be good to know your budget, I'll lay out for you a general list of locations to check out (from south to north):
Hastings-on-Hudson, Tarrytown,
Nyack,
Croton-on-Hudson, Peekskill,
Suffern, Stony Point,
Cold Spring,
Carmel,
Brewster,
Monroe-Woodbury, Chester, Goshen, Washingtonville, Middletown, Montgomery, Beacon, Fishkill, Wappingers
*The italicized locations represent those in Orange and Dutchess Counties. The northern borders of these counties are the unofficial end of what is considered the NYC Metropolitan Region (coterminous with NYC train service). Since these counties are a little bit farther from NYC, cost of living is a bit (this is relative) cheaper.
*Pink locations = Westchester County
*Red locations = Rockland County
*Orange locations = Putnam County
Schools are generally
very good in the region (no proof to back this up, but I'd argue some of the best schools in the Northeast are in the HV). Your children will have a good experience just about anywhere.
Shopping, retail, and dining is plentiful. You have nothing to worry about in regards to that.
The Hudson Valley tends to be quite artsy with 'creative centers' in many of the Westchester River Towns, Nyack, Cold Spring, Goshen, Beacon, New Paltz, and Rhinebeck.
Additionally, living farther north (say in Orange or Dutchess County) puts you along 84, a major east-west roadway with access to Connecticut and points east.
To give you some additional perspective, the following map shows the location of the counties that I've mentioned in relation to NYC. The fringe (blue) line depicts the rough end to the NYC Metro.
