Come to think of it a three pole switch is almost mandatory
isn't it,

Yes. 3 pole isolators for fans are widely available.

Christian.