I've seen caulk remover at the Borgs but never tried it. White vinegar
is good to clean up caulk before it sets, maybe it will help remove old
caulk. If nothing else it will remove any hard water deposits.