I have a gadget somewhere which fits on a hand-held drill and ensures it
drills centrally on the edge of a panel, and parallel to the long sides.

Pillar drills are not expensive and well worth having if you have the
space especially if you do much drilling. Almost essential if you need
to drill thicker metal accurately.