You don't *have* to remove it. But I would put it on your list of things to
do when you have spare cash - low priority.

I would remove it to keep cold air from getting in. Also may make more space
inside the house if it were removed.