What is the function of the JFET, Q553?

It looks like an audio switch that connects the L and R channels together in some mode controlled by the Mode 2 signal, for some reason. Is there a mono mode?

I'd disconnect this first to troubleshoot the other problem, it may be related or not.

Once you have the two channels isolated from each other, it may be easier to trace the distortion problem.

Mark