According to Abe :
The problem is that the neighbors don't see it. They have a fence built
up to the side of the shed, and the shed makes up for the rest of the fence.

That was a pretty critical piece of information to leave out until
now, dont'cha think? You could have saved some spinning wheels if you
had said this in the first place.

If it's a shared fence, you have every right to paint your side. Since
the back of the shed is de facto part of your side of the fence, you
have the right to paint it.

How do you know it's a shared "fence"? Unless it's exactly on the
property line, you can't really tell.

In situations like this, it's always best to ask.
--
Chris Lewis, Una confibula non set est
It's not just anyone who gets a Starship Cruiser class named after them.