Are the little cross shaped plastic tiling spacers you get supposed to be
left in situ and grouted over?

You can do either. You're supposed to put them in as corner pieces and grout
over. However, I find it easier to wedge them in the edges (2 per edge) and
remove before grouting. Otherwise, the tiles often ride up over the spacer,
wasting time as you try to remove it.

Christian.