Did you not mount the "union" to a brace between the studs ?
usually you use a cast fitting with 2 screw holes.

but can you get to it from behind, or is this against the basement wall ?

I braced it enough so that someone playing with the showerhead couldn't
easily move the pipe - but the bracing wasn't intended to prevent twisting.
Honestly, it never occurred to me that I might have to remove this thing.

Anyway - I went for broke and took off the metal shower stand. Nothing
inside the wall seemed to break, even though I had to twist quite hard.
After applying a lot more teflon and remounting it - the damn thing still
leaks, only not as much. =)

I think I'm going to give the caulk a shot. Heck - the stuff only has to
hold up to a five minute shower a few times a year.