From the Mercer Co. area (Princeton, Robbinsville, West Windsor, Hamilton) your commute will be 1-1 1/2 hours away (public transport) to Manhattan and your husband will be 45 mins to an hour away (driving) to Southhampton.
Look at NJT.com and the northeast corridor and you will find the towns with access to the stations that might make your commute shorter