Errm, anybody had any experience of ground nesting bees?

I'm clearing out, and demolishing, an old shed. On the floor was an old
door with a little hole in it where the bees live. I shifted the door, and
disturbed about 6 or so bees. Could there be a few hundred more further up
the shed?

Are they likely to just bugger off shortly or am I going to have to get
someone in to clear them out.

Any help appreciated.

Mal.