Thanks for everyone's help. Had an electrician come out and diagnosed
what was going on. The switch was wired fine, I think he called it back
loading?, but the issue was one of the junction boxes where all the
wires came together. The people who had done the work originally had
mixed up a few wires, his technical term was "it's a mess", but he got
everything working great. If I recall correctly now, there was a
nuetral connected to the ground wire and that is what caused power even
with the switch off.

Also, at the end of the day, he recommended I get a digital tester,
which was wonderful b/c his reco helped side step the fight with my
wife when I told her how much it was!

Thanks again for everyone's help.