"Some drivers don't like it and warn of danger ahead, including mud, dust and damage to their cars."
Ohhh, dust, now that's scary.
At first, that doesn't sound like a good idea, but if a road has little traffic, gravel will suffice. Some roads in my area, southern New Jersey, have only been paved within the last few years. One runs through a wildlife management area; people now drag race on it. Gravel was better.