I spent a lot of time traveling in Europe over this past year, and we spent the xmas/new years holiday in Budapest, Vienna and Prague. My wife is currently living in Budapest for work, so I spent most of my time there.
Prague was my favorite of those 3 cities - I loved the fairytale architecture, it reminded me of scenes from Shrek or medieval movies. I found it to be extremely clean as well.
I've never had any issues with theft in any of these cities - my wallet does stay tucked away in a close fitting pocket or a pocket beneath my zipped coat, but I carry around my very expensive camera and lenses in my backpack without any troubles.
Don't bother driving and do plan to use public transportation. Trains go all over Europe and are extremely convenient. Consider overnight trains as well - you fall asleep in one city and wake up in another, it's a little noisy and tough to sleep well, but it's a great way to knock out transportation and hotel in one. The metro and tram systems are also great in each of these cities.