The spring and fall seem to be times when neither the cooling or
heating systems are running in a lot of houses ( depending on local
climate of course ).

In that case, you may get stagnant air in the house, 'tight house'
syndrome mentioned above. Try leaving a window or two open and see if
some increased air exchanges makes the problem any less for you.