A false positive is not the same thing as a flawed test. All test
statistics have statistical properties and when you pick a threshhold
to indicate positive, you are picking a probability you'll indicate
"positive" when in fact the true state of the world is "negative" -- a
false positive. OTOH, a "flawed test" simply doesn't provide
information about the subject of interest, like your absurd garden hose
example.

Consumer Reports is not going to provide us with any more detail than I
gave on the test. Beyond that description, you either trust them or
you don't, as you see fit. I tend to think there's actual information
in their ratings. They have a lot to lose if they construct a flawed
test. I don't recall seeing window companies objecting to the
evaluations when I googled "consumer reports and windows". They do
note in the article that window "makers participating in a program run
by the Insulated Glass Certification Council subject insulated glass to
tests that expose the glazing and its seals to heat, cold and water.
Glazing that passes three test segments earns a "CBA" mark, etched in
the corner of the glass." Sounds like that industry group evaluates
glass and seals using a method similar to Consumer Reports. Sounds
like you don't trust them. So be it. Take what you like and leave the
rest.

Cheers.