I am fairly sure it was from the installers, not the factory guys.
Maybe it is some residue from the caulk, but I would have thought that
the goof-off would have taken care of that, if not the turpentine.
After 15-20 minutes of scrubbing and rubbing it looks clean, and is
only visible when the window is wet, foggy, or the sun hits it at the
right angle. So I'm not worried about those situations (no OCD), I'm
more concerned with having to vigorously scrub 19 more windows for 20
minutes to achieve satisfactory results. Especially when half those
windows are on the second story and I'll have to half hang myself out
the window to be able to get any leverage.

Thanks for the other suggestions - I'll try lacquer thinner and
gasoline and just be careful about where I get it.