wrote

So the question is, is it worth it (energy wise) to lower the temp
during the day, when it takes so long to get back up to temp? I'm
thinking I'm buring as much gas to bring it back up than I might save
during the day.

Energy wise, the amount of heat you lose is directly proportional to the
temperature difference between the inside and outside of your house, so
setbacks do work. If it's taking too long to heat up your house, that is a
different issue. Either your furnace is too small or it needs fixing.