Check for a dripping water heater relief valve. It will increase water
bill as well as gas bill.

Check for an open fireplace flue.

Make sure all rooms have returns or leave interior doors between rooms
open as much as possible to provide a return path for the air.

Is it more comfortable and less drafty with the new windows? If so,
then something else is causing your problem. Have someone do a blower
door test on your house.

Stretch.