I've got a piece of electronics on my bench that is normally powered by
3 C batteries, wired in series and thus producing about 4.5 Volts. I
wanted to power this device with a bench power supply just so I don't
drain the batteries while I'm experimenting with it for hours.

But it won't work with the DC power supply -- it's the weirdest thing.
With the batteries, the device (a child's toy) chatters away at full
power, but if I disconnect the batteries and wire in the DC PS in their
place, it runs veeeeery slowly, if at all.

I feel like it's the 18th century and I'm missing the right amount of
"essence" or "ether" or "humours" or something

I've checked and double checked the voltage including polarity. I've
tried running at depressed voltage (e.g. 3-4V). The power supply
should be able to provide far more current than the batteries, and at a
low output impedance. I have two power supplies, one a fancy bench
model and the other a wall wart, and it does the same thing with both.
Only with the magic batteries will it work properly.

Perhaps related, in the device at the + side of the battery back is an
orange disc which appears to be a ceramic capacitor wired in SERIES,
but I don't see how it could be given that this is a DC device. I've
been attaching the DC PS on the battery side of this disc, in order to
provide DC power to the circuit just like the batteries. I don't know
if this is relevant.

Any ideas?