Tim+ pretended :
To complicate matters theres another soldered fitting just over an inch
away that I really dont want to disturb (so may end up packing it in ice).

Any tips on how to cleanly remove solder to eliminate the risk of my olive
not seating/sealing properly?

Wrap the one you do not want to disturb in wet tissue and apply the
flame only to the other fitting. Have a spray bottle of water handy, to
also cool it down. Stop the heat as soon as it moves.

Course emery cloth will remove the solder from the pipe.