4mm would do, 6mm would be better. There's no significant load on the
threads, the threaded part just hold another minor part in place.

Inserts would stick out the other side and I need the front & back to
be flat so that it can slide into alternative fixing positions - where
it's fastened by a bolt through a hole into something more solid.

I don't have a heavy enough bench for a decent vice.

I do have a tool for setting threaded inserts. I don't much like it as
the inserts don't always bite very well and sometimes work loose enough
to rotate in their hole. The tool looks much like the one you linked
to. I get better results by doing what I previously used to do. Cinch
nuts, a ball bearing and a hammer !