I have a requirement to drill brass set screws with a 1/64 drill bit.

Jacobs makes a nice Keyed drill chuck with a MT0 arbor that will do the job.
I think its a 5/32 chuck. They are frequently on sale for about $45.
It is a very short chuck which was really nice when I used it on my sherline
mill. A 1/8 Albright is good too, but its about twice as long and costs
over twice as much.