Thanks to all of you for your input.

I will use brand new drill bits, possibly with a pilot point. I will
locate the holes using coordinates and start each hole with a center
drill, allowing it to hover in the hole so it centers well.
Your comments about steady feed and pecking will help.

I don't think I will go to drill bushings, though. Just too much effort.
I would like to eventually make several of these tools.

Thanks again,
Pete Stanaitis