How about doing this?

Start with the edge finder approximately in the center of the hole. It
doesn't have to be exact. Move in one direction along the X axis until you
find the edge. Zero your DRO. Move in the opposite direction until you
find the edge. Note the reading. Move half way back to zero. Zero the
DRO. That is your zero for the X axis.

Then do the same thing in the Y axis.

The reason this works is that regardless of where you are in relation to the
actual center of the circle, you are traversing a chord of that circle. If
you bisect the chord and draw a line perpendicular to it at that point, the
line will go through the center.

This should get you accuracy as good as what you are doing and take a lot
less time. Just how accurate do you need to be?

And of course, all this assumes that the hole is actually round.