Jon wrote:

Maybe I'd be better off insulating the garage. But the garage door has
so many gaping holes letting cold air in through the gap between the
frame that it'd not be cost effective either unless I changed the door
(expensive).

I'd be inclined to do the garage. OTOH if it's particulary small, you'll
probably not be able to open car doors when you've lost 6" or so off the
walls for insulation. Could you not use foam tape around the door?