Absolutely insulate the ceiling. You might want to close it off for a
couple of weeks in the summer and see how warm it gets.

There's a book called something like how and why to build a wine
cellar that's well worth buying. Less than the cost of one bottle of
wine, unless you're drinking $4 wine.