try using Great Stuff spray foam. Just be carefull how much you use, it
expands uncontrollably. Test it first.

Note, they do make different kinds of this foam. Using just the 'regular'
foam may not be a great idea. It expands QUITE a bit. They make another
variant that expands with less pressure. This is the sort you'd use around
windows. If you used the regular stuff you run the risk of having it expand
such that it causes the window to bind.

When insulating an outlet be sure NOT TO PUT THE FOAM IN THE WIRING BOX.
The foam is permanent, it does not lend itself to ever being removed.
Certainly not without wrecking whatever it covered. If you put it inside
the electrical box you not only ruin any chance of ever fixing the wires,
you may also be introducing an added fire hazard. So DO NOt spray it into
the box that contains the wires and the outlet. And if/when you did spray
it around the outside of the box be sure to NOT use too much such that it
squeezes INTO the box.

That's why I suggested NOT using the regular variety. If you use the less
expanding foam most likely the worst thing that'd happen is you'd end up
needing more of it. I'd rather use another can instead of having one expand
too much, become impossible to remove and ruin things.

Whichever foam you use be prepared to use the whole can, right then. They
don't keep. Get all your outlets prepped before starting. Along with any
other cracks leaking air.

Read the instructions on it carefully and be prepared for a mess.

-Bill Kearney