I'm going to be replacing my windows soon. I plan to use minimal
expanding foam to fill the gaps.

The Great Stuff for windows (blue can, "Will not bow frames") says
uncured foam can be removed with acetone. Can I assume that the
acetone will not harm the vinyl?

Acetone CAN dissolve many plastics. I wouldnt use it. I have "lacquer
thinner" in my garage and generally use that for such jobs..it also will
affect some plastics.

I've been practicing trying to neatly fill a 1/8" gap with the
supplied straw and it's pretty tough, even after pinching the straw so
it will fit into the gap.

Instead of the "Great Stuff" YELLO expanding (horribly sticky/messy) foam,
for this 1/8" narrow application, I used a WHITE latex foam (Brand is DAP I
think) in a can and it wiped off easily with a rag.

Do I need to put foam in a 1/8" (or smaller) gap?

Yes, you should...seal ALL gaps