Thanks for the great replies!! Very enlightening.

I'll take apart the misbahaving UPS and check the
batteries. If there's one that's a lot worse than the
other 3, I'll probably replace it. If it's just a little
worse than the others, I'll try to get a smaller (like
1/3 - 1/2 capacity lead-acid) and parallel with it.
If all are equally bad, I might just need to bite the bullet.

I know that replacing/paralelling one is not a proper
thing, but if Matthias is right, the UPS probably has
1 bad and 3 OK batteries. When I have 3 bad ones,
I'll replace them all. I have another one of the same
UPS, so I might use those extra batteries left
over when all old ones are replaced.

I'll also look into a few other things when the UPS
is open, like slightly decreasing the charging voltage,
or making it more tolerant to marginal batteries.
Assuming I can reverse engineer,or understand the
circuit, which could be very difficult if it uses too much
custom, or VLSI ICs.