I would try applying mineral spirits to the flagstone surface, and then wicking up the mineral spirits with paper towels.

I'm hoping the oil dissolves in the mineral spirits, and the mineral spirits are absorbed in the paper towels.