Dap Seal n' Peal is the stuff I have seen at my local Lowe's store. I
almost bought it one time too but didn't want to spend the money
knowing that I would end up removing the stuff the next season.
Besides, that, I didn't know that I would find all the air leaks.
Instead, I bought what is basically large sheets of plastic wrap that
is stuck to the windows with a very clear film/tape. You basically put
it along the outer portion of the window, stretch the plastic on, and
use a hair dryer to shrink wrap the window. It comes out looking great
and you don't feel any drafts come in off the windows.

I did it on all the windows on the bottom floor of my house and I have
been a lot more comfortable (so has my heat pump since it is cycling
yet). My wife isn't much more comfortable but she would be cold in a
desert.