Yes, those 6 ft. wood "privacy fences" are just an interesting feline jungle gym...
The two options as I see it would be to either a) erect a cat-proof fence inside the existing fence.... there's a specific one I have in mind and the brand name escapes me. It's very wiggly so the cats are not able to gain a purchase and therefore can't climb - our local cat shelter has the garden fenced with it so the cats can play outside. The one there is a good 6 ft plus the angled-back top.
B) would be to somehow or another attach the angled-back supports to the top of the wood so that even if the cat gets to the top, it can only go back into your own garden.