Rhinebeck and Red Hook are both quaint, small towns, as is Millbrook. You may want to consider Staatsburgh as well in your search for homes. I believe parts may go to Hyde Park schools, however, not Rhinebeck.
Hyde Park school district, in the past (I've been gone 5 years, so I'm not sure what it's like currently), historically had problems getting the school budgets passed. They were, however, a larger school than Rhinebeck school district.
Rhinebeck schools are good from what I remember. They were 2nd on my list of "small" schools, with Spackenkill being tops in that category.
Some sites to check out some school info at:
NYSED:IRS:Schools and Districts In Need of Improvement (http://emsc.nysed.gov/irts/accountability/sini/ - broken link)
GreatSchools - Public and Private School Ratings, Reviews and Parent Community
You can get around the Dutchess County Fair area pretty easily as long as you avoid the mayhem around 9-10pm when everyone starts to leave.
You can still hop on a train and be in the city in no time, too!