My neighbor has a fence and the back side faces my backyard.

Can I paint it to match the color of my house, or do I need his
permission first?

Thanks.

Corinne