Jim Thompson wrote:

Unplugging the USB connector seems to work. So why wouldn't a switch?

The outer two pins of a USB connector carry VCC and ground while the
inner two pins carry a differential pair. Put a switch on either line
of the differential pair.

--
Don Kuenz