A smarter man would've saved the wood for burning, as the coming winter season, and the rising cost of fuel, will bring issues far greater
than those snooping town inspectors and new fences. I could be wrong, (but rarely am), and can't see that if the fence that you recently replaced was
legally there in place, why would it be an issue with your town when, and if, you put up a new one? If, when you bought the property, there was
a fence in place, why would the authorities even care if you replaced it? Perhaps there is more to this story than is currently apparent to you.