Now we have more detail it suggests various options
1. speak to the owner of the other side of the fence and access from that side
2. get help for the first attempt while you install stainless steel eyebolts at each end of the restricted section with a tensioned stainless steel cable between them to which you can secure a sliding uv resistant nylon rope.
Whenever you require to access the gutter etc place the ladder and slide the rope next to it. Hold the rope under tension as you climb to prevent the tendency to fall backwards and when you are high enough tie it off to the ladder. On completion reverse the process.
3. Install "hook" brackets to engage with the top rung and "hang" the ladder before climbing.
4. Buy one of the compound ladders which have sections that can be conjured to form a big FO step ladder and work sideways. (The neighbours will probably want to borrow it from time to time)