Pressure relief valve
 

A pressure relief valve at the main water valve is left over from the
days when they were not required on water heaters. As the water heater
heated the water and it expanded it would pop the relief valve rather
than blow up the house plumbing.

These days water heater manufacturers must install relief valves right
on the heater. Plus, many communities require an expansion tank near
the WH to absorb the expansion.

Some cities also required a one-way check valve to prevent the
expansion from forcing your water back into the city's water supply.
That prevents your house from contaminating the entire community if
something goes wrong at your place.

You have two choices depending on what your city requires these days.
You can either remove the valve and cap off the opening. Or you can
replace the valve with a new one. Of course, that assumes that you
already have a relief valve on the water heater.