In my garage there is a felt material in the expansion joint. It is
20 years old and dirty and rotted. Is there anyway I can pull this
felt out, or cut off enough enough material such that it is low enough
for me to fill in with a caulking?

I have already filled the other joint running across with caulking,
which has no felt material in that one.

The reason is I have coated my floor with epoxy and cosmeticly the
rotted felt looks bad now, plus I wanted to minimize the insect and
moisture crawling up.