I prefer Murray's Bagels to what is now H&H. H&H used to be a bit better, especially when buying from the factory on 46th, as I do now when Murray's is closed as H&H is open 24-7.
Murray's Bagels
If you like cheesecake, S&S is very good, but in The Bronx, and whole cakes only. You can get selections at Zabar's, however, as well as numerous other delicious treats.
Cakes and Cookies @ Zabars
For desserts, Glaser's Bake Shop and William Greenberg Jr. Desserts are wonderful. They are regular bakeries, so you would find cookies, black and whites, babka, cakes, cupcakes, etc.
Cake Bakery New York City | Cake Bakery NYC | Glaser's Bakery
William Greenberg Desserts
For doughnuts, I would recommend skipping the numerous international chains (not because they are inherently bad, but because you can likely get those at home). Instead, I would recommend a visit to Doughnut Plant.
Doughnut Plant