You really need to get a real tester. With the switch off, the hot should be dead.

What is a real tester? I did go buy a little pen like device that beeps
when it is near voltage. It beeps near the ceiling fixture with the
switch on or off.