Your solution on this would be to replace the fan with a non timer
fan. I dont know if you could even get away with just removing the
constant supply wires from your existing one so that it is only being
fed directly from the lights supply. No doubt someone will help with
this. My gut feeling is that by simply removing the constant feed that
this will solve the problem.

Be aware though that even with the light switched off, there will
still be a live cable at your fan, so I would advise turning off your
electrics before doing this job, and then just putting terminal blocks
to isolate the cables that you remove. If at all in doubt with your
electrics then call out an electrician. If you feel competent then
keep the electricity turned on, put your old switch in. Put it in the
off position and then use a multimeter to ascertain which cable/s are
the constant feeds.

Good luck and stay safe
Calum Sabey
NewArk Traditional Kitchens 01556 690544