I am very much leaning toward the ER32 solution, but want to read a

I have an ER32 bison collet chuck that I use on my horizontal mill with
a #30 spindle. I used it to hold small endmills (less that 1/2) and
to hold woodruff key cutters. It works wonderful for both. (I also
have end mill holders but carbide endmills dont have flats)

My first big mill was a jet mill drill. Switching from drilling to milling
without moving the head was tough. I bought an Erickson DA180 collet
chuck for it. Double Angle (DA) collets are primary made for drilling
but I have used them to hold small endmill without any problems. They
also make special collets that with an NP postfix intended for holding
endmills. These NP collets have a button that engauges the flat on
the endmill. Read all about them in the J&L catalog.

Beware that both systems are pricy. I paid about 200 for the Bison ER32
collet chuck from Penn Tool. I think the erickson 180 cost me double that
because the collets are expensive. I bought mine on sale from Enco.
Now that I have the collets, I have various chucks to use them in
various ways. R8 collet chuck for the mill, MT collet chuck so I can
use them in the lathe tail stock. Another big plus with the 180 collet
chuck is that it will run about as true as you can get a drill to run.

chuck

chuck