I have a CR2032 mercury battery, and when measured with a digital VOM,
it starts around 3.5v and climbs slowly to a little over 6v .

What does that mean?

It's a cheap meter but it's worked fine for years, especially for DCV.