I believe I can manage both.

Because a red photon has more energy than an IR photon, the IR sensor would
likely respond to the red light from the "other" remote. So, no, you couldn't
easily isolate them.

You'd probably need fairly sharp optical bandpass filters. Is it worth the
trouble?