....
disintegrated. So what is the right tool/method just to widen those holes
on one side?

The right method is an endmill in a milling machine. Bet you ain't got that.

Drill a 1/4" hole in a fairly thick piece of scrap stock. Clamp it to the
piece you want an offest hole in. Use the scrap stock as a drill bushing
guide to hold the drill on the new center. Feed slow this is hard on drill
bits.

Karl