Yes. LOL. The Yorkville area in Toronto safe (unless Holt Renfrew, Prada or The Louis Vuitton stores decide to have some epic crazy clearance sale
!). Toronto in general is pretty safe compared to the other places you have mentioned. This area is a good location to stay in as it is centrally located and you will be able to get around the city easily by foot or public transit. Hope you have a great time in Toronto. Best wishes.