Stuck contact of a key?

The obvious one would be the Play button (as the unit starts playing when I
put the batteries in).

I'll try holding down the Play button on a "good" unit, and see what happens.