Welcome to the club......
Basically it was a form of what is known as skip troweling. You are probably describing the pattern known as Mud Swirl.
It was just old normal drywall mud, they can get tricky and add different things. Some are fine sand, latex paint or other texture additives. Many patterns are done with a trowel of some type. Some of the swirl patterns with deep grooves they used a notched trowel insteal of one with a smooth edge.
The swirls are created by rotating the trowel in a circle and then going back into the pattern a bit to start the next. Can get very creative by using different tools, brushes, stamps, rubber tools, etc.
Problem being many of those patterns are a bear to repair if the need ever arise. Lots of luck if you have the problem. Especially those where they did the color by adding paint to the mix. You never get that puppy to look right.
Even a bear to paint over it and get it to look right. High on my list of things never to have in my house. Your type of major repair where it is best to remove the crap and start over.
One of my sisters had a house with the swirl pattern using fine sand. Duh, impossible to touch up.