Hello and welcome (potentially) to the area! You are drawn for the right reasons; the prices of homes are definitely cheaper than Nassau/Suffolk, more land/open spaces than Nassau/Suffolk, and it is still in commuting range.
Generally, you want to avoid the City of Newburgh, Middletown, and Port Jervis.
For places to live:
There are many attractive locations in the county that are worth your time in investigation.
Cornwall
Trulia
Wiki
Picturesque town in the eastern part of the county, 5 miles from West Point. Right on the Hudson River. Good schools. Major retail a short ride away in Newburgh or Harriman. Palisades Parkway very close and 87 just to the west for all points south.
Montgomery
Trulia
Wiki
Small and affluent village just inside the eastern crest of the Catskill Mountains. Superb schools (Valley Central School District). Grocery store (Shoprite) and other stores in-town. Major retail 10 miles away in either Newburgh or Middletown. Plenty of parks nearby. Right on I-84 and near 17/87.
Washingtonville
Trulia
Wiki
Another small and upscale village. Excellent schools. Stop & Shop in town. Major retail a short ride away in Newburgh or Harriman. Plenty of parks nearby. Great schools.
Goshen
Trulia
Wiki
The county seat of Orange - one of the most expensive spots in the county. A nice village with excellent schools. Major retail 5 miles away in Middletown. Plenty of parks nearby. On Route 17.
Chester
Trulia
Wiki
Village in the southern part of the county. Great schools. Good amenities. Right on major Route 17. Plenty of parks nearby.
Some other locations that are worth looking in to: Monroe/Woodbury (
Trulia,
Wiki), Central Valley (
Trulia,
Wiki), Warwick (
Trulia,
Wiki), Florida (
Trulia,
Wiki).
You could also look into options on the other side of the river, such as Beacon, Fishkill, and Wappingers Falls. You could have a look at Cold Spring, too.
Other Orange County Info:
*Major retail in the county is in the Town of Wallkill, Town of Newburgh, and Harriman.
*Many commuters to NYC, but I wouldn't call any community a 'bedroom' community.
*Rail service to NYC is abundant (stations at Port Jervis, Otisville, Middletown, Campbell Hall, Salisbury Mills/Cornwall, Harriman, and Tuxedo with service to Secaucus). Other option would be direct service to Manhattan out of Beacon. Buses also plentiful.
*Second fastest growing county in the state according to the 2010 census. Growth started in earnest following 9/11.
If you have any specific questions, please let me know!