I even added DE powder to my sand filter to see if it was just missing
some "algae".

If your sand filter is broken, sand will trickle out and collect in the
strainer in the Polaris booster line where you disconnect. Water will
still flow through the line past the sand (like it does in the filter), but
disconnecting will flush out the sand into the pool. This "little sand
filter" will also trap dirt. This would be consistent with your "brown
cloud" description; you're sort of "backwashing" (flushing) this dirty
little sand filter into your pool. Adding DE to the (regular) sand filter
will not stop this sand leakage, as it is happening at the bottom, not the
top, of the sand filter.

You could collect the "brown cloud" into a garbage bag and decant it for
sand, to diagnose a sand leak. Or empty your sand filter and inspect the
strainers.