Caulking in our bathroom, around the base of the tub & floor, and
around the floor/walls in general, is either cracking or is missing in
areas.

I read up on this before (in our previous place, an apartment) but
never followed through since we were leaving for our new place (first
house).

Correct me if i'm wrong, and please feel free to inform me further,
but i think i need to remove the existing caulking with some kind of
solvent? Then clean & dry the area, the apply the new caulk and let
dry.

Is that right?