I'd probably take the collet to an auto machine shop and have them hone it
to the desired size with an abrasive hone. You'll want to put the collet in
a holder and put some brass in the slots so that it doesn't compress.

--
Why do penguins walk so far to get to their nesting grounds?