You mentioned having a solid fuel heater, which is the
traditional method of heating UK homes, but does not
seem obviously responsive.

I'd have a combination of solid fuel and oil. The oil would get things
going fairly quickly and could be set to come on in the early mornings
when solid fuel would not be an option. I'd use solid fuel during the
day.