I had to look up digger wasps to see what they were. Cicada killer wasps, those gigantic and menacing looking buggers that make their nests underground and appear this time every year.
Believe it or not, they are harmless to people. We field this question on the Master Gardener hotline every summer. Their stingers are too blunt to sting us or our animals. All they do is kill cicadas. So if you decide not to remove the nest (my recommendation), you don't need to worry. I figure if they don't harm us or our pests and don't destroy our gardens, let them be. We have a news in a bed outside our front door, and it can be annoying, but I'm not removing it.
Cicada Killer Wasp and Other Digger Wasps | Iowa Insect Information Notes