Don,

Destroying your neighbor's property is not legal. It's not clear in your
posting why you and the neighbor aren't resolving these issues though it
sounds like you see mountains where others see mole-hills. Here are some
options.
You may write to your neighbor giving your permission for his encroaching
fence. This may prevent "adverse possession" of the property and may help
heal whatever ill feelings exist.
You may complain to various municipal inspectors about the fence. There
may be a need for a building permit. There may be a need for a setback.
There may be a requirement that the "good" side of the fence face outward.
This shouldn't be expensive.
You may sue in civil court (this is not a small claims case}to get an
order to fix the fence. You'll probably need a lawyer to do this.

Dave M.