ccsikyr wrote:
A false positive is not the same thing as a flawed test. All test
statistics have statistical properties and when you pick a threshhold
to indicate positive, you are picking a probability you'll indicate
"positive" when in fact the true state of the world is "negative" -- a
false positive. OTOH, a "flawed test" simply doesn't provide
information about the subject of interest, like your absurd garden hose
example.

It was meant to be absurd. It was meant to show that testing a system
for something it was not designed for is not testing the system. I
said that.

Consumer Reports is not going to provide us with any more detail than I
gave on the test.

Okay...I haven't read the report. You have. I'm not sure why a
testing procedure wouldn't have its specifications listed. Two week
duration. That's a specification. Intermittantly sprayed water is not
a specification - it's a very loose description.

Beyond that description, you either trust them or
you don't, as you see fit. I tend to think there's actual information
in their ratings.

Of course, but the question is, does the information have relevance for
what I need a window to do?

They have a lot to lose if they construct a flawed
test. I don't recall seeing window companies objecting to the
evaluations when I googled "consumer reports and windows". They do
note in the article that window "makers participating in a program run
by the Insulated Glass Certification Council subject insulated glass to
tests that expose the glazing and its seals to heat, cold and water.
Glazing that passes three test segments earns a "CBA" mark, etched in
the corner of the glass." Sounds like that industry group evaluates
glass and seals using a method similar to Consumer Reports. Sounds
like you don't trust them. So be it. Take what you like and leave the
rest.

I wrote: "I think Consumer Reports is a great publication - a great
service. But unless you've edited out a lot of information in your
synopsis, the results are not relevant as the test is just bizarre." I
don't see how that could possibly be construed to mean I don't trust
them.

I was merely asking what "severe temperature fluctuations" means. It
may mean 30 below zero to 150 above in an hour - I have no idea. Both
ends of that spectrum are meaningless to me in my climate. Windows are
designed for longevity, trying to force a test to have failures (how
else could you determine the winners and losers?) in two weeks means
something fairly extreme is going on. I'd like to know specifically
what the extremes are. I don't think it's really asking a lot if I am
to give credence to anyone's testing procedure. The "intermittent
spraying with water" also needs clarification. Spraying doesn't
necessarily indicate anything at all about the amount of water and the
pressure.

I don't take anything on blind faith, and I don't happen to feel that
CR is an unquestionable authority. I don't subscribe to CR, so this is
all I found in a quick Google:

"How to choose
Performance differences. Consumer Reports has found most windows do a
very good or excellent job at sealing out a fairly strong wind when the
outside thermometer registers 70° F. Only a handful do well at sealing
out a high wind when the outside temperature drops to zero. When it's
that cold, weather stripping and other components can stiffen or
shrink. Our tests have shown that aluminum frames are durable. But we
have found windows with frames made of vinyl- or aluminum-clad wood can
perform well, too.
Recommendations. If you're replacing windows, choose those that are
designed for your region's climate. Cooling costs predominate in
southern regions, so look for double glazing and a low-E coating. Give
first consideration to windows with a low solar-heat-gain coefficient.
The Department of Energy recommends that the number be 0.4 or lower."

There are specifics in that. 70 degrees vs. 0 degrees, for example.
I'm more interested in the 70 degrees as the 0 degrees is a lot rarer
around here. My personal experience also counts for something - at
least to me. If I hit an aluminum clad window frame with a bat, in any
weather, that dent will be there forever. If I hit a vinyl clad window
with a bat on a 70 degree day, it won't do much of anything, while on a
0 degree day the vinyl will almost certainly crack (don't hit vinyl
windows with bats in very cold weather).

R