RBM wrote:

The picture isn't quite clear enough to verify it as a zone valve as opposed
to a solenoid valve. I've never seen one built right into a baseboard
heater. If it is a zone valve, it'll have a couple of springs inside the
cover that force it closed when the thermostat is down. Possibly they are
catching on a piece of sheet metal of one of the heaters? Zone valves close
fairly slowly, at least compared to solenoid valves, so they usually don't
cause a water hammer

I opened it up:
http://mikkopel.dyndns.org:1000/misc/baseboard2.jpg

I had my wife turn the heat off, while I was looking at it, and I can
see the motor action when it happens. But now it's not making the noise.
I'm not sure if it's because the cover is off, but then again it didn't
make the noise every time anyway.

There also seems to be a lever of some sort on the left of the picture.
What's that for?

Doesn't look like it's ever been cleaned under there either...