I've never lived in Albany, but worked there 1/3 of the time. Most of my colleagues lived outside the City; just a few lived in the City, and if I recall correctly they were either senior management or single.
I'd generally stay in a great B&B in the Mansion District (in walking distance to the Governor's place and my office) but still considered iffy. I didn't find it iffy because I lived in NYC and none of the Albany neighbourhoods really seemed more dangerous than those I was walking in regularly.
One or two neighbourhoods were really appealing--townhouses, good restaurants, some fine looking shopping, but other than that I think Albany doesn't seem to offer what the immediate suburban areas seem to offer the folks I know. It has a very bad reputation among those I know and we are talking hundreds of folks not a few. I think its reputation is, like other cities, based on high crime, extreme poverty, loss of industry, etc. I did meet one cab-driver (they are the best judges) who took me on a free guided tour of Albany's past!
And yes, Saratoga is right up the way, and in fact, several times I'd stay there for the ambiance and take the bus into town, or get a co-worker to pick me up early. Saratoga is among the nicer small towns I know, but it is expensive. Many people commute as much as 1hr/10 minutes from places like Greenwich and Balston Spa rather than live in the City.
That's all I know! Good luck and hope you have an idea of a good paying job before you head out.