I was thinking of putting a little glue along the edges of the panels
to stop them from moving.

It won't work. The glue will probably just break, and if it doesn't, the
door might instead. Something has to move.

This is probably more trouble than it is worth, but wood only
expands/contracts across the grain. If you can center the panels in the
frames you can glue the center; that will allow the top and bottom to move
independently. It will significantly reduce the overall movement.

Now, it will be difficult to glue just the center, and it will not eliminate
the problem; but I think it is probably the best you can do. (other than
stripping off the paint and staining it.)