Can anyone here confirm that this only occurs if you use oil based
paints? Meaning that latex will stick to galvanized fi you just clean
it.

Having said that, I thought that cleaning and sanding (more like
scuffing) is what one needed to do to get (oil based) primer to stick
to new galvanized steel.