The procedure Jim Rozen describes appeals to me because it uses the kind of
shop tools I have, rather than special drills, laser guided center punch,
and so on. Here's another technique I have found to work for me, but I
don't pretend it will produce "absolute" accuracy.

I scribe two lines at right angles through the center I want, and center
punch at that location. I then start drilling to a size smaller than the
finished hole size. Chances are the hole will start a little off-center,
for the reasons cited by others, earlier. I chuck a drill bit in a hand
drill, and drill at an angle until the pit I am making looks visually
centered on the scribe lines, and then continue with the drill press. I
can't put a number on it, but your eye is pretty good at judging the
centering of a small round dimple.