Sounds like Sierra Vista, Arizona to me.
As I've mentioned before on this board, I think SE Arizona has the 2nd best climate of anywhere in the continental US (2nd only to the coastal areas of Southern California).
Most folks think that all of Southern Arizona is furnace, but the fact is, this is simply not true. What people don't realize is that the climate is the SW is largely driven by the altitude and that as you head east from the California border, the average altitude in Sourthern Arizona begins to climb. Thus Yuma (in the extreme SW corner of Arizona) is at 200 feet or so (it sits on the shores of the Colorado River) - and is EXTREMELY hot, while Phoenix is at 1,000 feet - and still VERY hot, Tucson is at 2,400 feet and still pretty darned warm but Sierra Vista is at 4,200 feet or so and MUCH more temperate.
Average highs during the hottest month for Yuma top out at 107, Phoenix has about the same, but Tucson tops out at about 100/101 (still pretty darn hot admittedly), but Sierra Vista tops out at 93 or so (definitely pretty hot, but not nearly the furnace that places further west have to deal with). Plus the winters are sunny and very mild (a favorite spot for hummingbirds during the winter months) with highs typically in the low 60's or so.
Furthermore, the rainfall also increases as the elevation goes up, so whereas Yuma and Phoenix get hardly any rain at all, Sierra Vista gets 14-18 inches or so a year. Not an enormous amount of course, but enough so that the desert landscape is starting to change over to grassland with some trees. Take a look at their average temps over the course of the year:
Average Weather for Sierra Vista, AZ - Temperature and Precipitation
As near as I can figure, other than boating (nearest boating lake is probably Patagonia Lake) SV has every single thing you mention -
Mild Climate - (Average summer high tops out at 93, average January low is in the low-mid 30's (with an average Jan high around 60))
Dry Climate - High Desert/Grassland. Lots and lots of sunshine year-round.
Smaller Town - Population 42,000 with most everything you need - plus it's only 90 minutes to Tucson (pop 1 million).
Low Crime - Lower than the US average
Recreational activities - plenty of mountains around for hiking/camping, 3 golf courses in SV, others in Benson and Tucson, VERY scenic area with lots of sights to see (Tombstone, Bisbee, Karchner Caverns) in the immediate area and LOTS more things a bit further out (Sedona, Tucson, Phoenix etc). AMAZING views from Sierra Vista looking SW into the Huachuca (pronounced "Wah Choo Ka") Mountains, south into the mountains of Mexico or east down across the San Pedro valley - you can see FOREVER. Plus only a few hours drive to the Mexican beach resort town of Rocky Point (Puerto Penasco).
Find out more here:
http://www.city-data.com/city/Arizona.html
Ken