So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on an
unreliably small piece of evidence and get the same result as the first
time, this strengthens the case that the original result is correct and uses
some coin-tossing explanation to "prove" her case.

I disagree, her corroborating example repeats the same tests on *different*
data whereas doing the DNA test again would be repeating it on the *same*
data (and as such, is completely worthless extra information).

I think that she has fundamentally missed the point about the unreliability
of the source data here.

Anyone any better ideas than me, what is the right answer here?

(I should really be posting this to ul, but I think the science experts are
in this group?)

TIA

tim

On 29/04/2013 21:24, tim...... wrote:
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data (and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.

Anyone any better ideas than me, what is the right answer here?

(I should really be posting this to ul, but I think the science experts
are in this group?)

TIA

tim

Well, presumably the first test was on a small sample taken from the
knife, and the proposed repeat test would have been on a different
sample. And that, I think, would improve the statistics although without
knowing the exact methodology it's difficult to be sure.

But it was the scientist suggested a repeat test and I am inclined to
think they would have understood the statistics. The journalist's point
was that when courts get the science wrong, people suffer. For example
the Guilford "bombers", the "cot death" lady, the Dutch nurse...

On 29/04/2013 21:24, tim...... wrote:
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece


Before we follow this line of thought, I observe from the BBC report that:

"the DNA sample was tiny, and the appeal judge thought the
evidence was unreliable, so he rejected a forensic scientist's
suggestion to have it tested again"

which suggests that the "unreliability" of the sample is the subjective
opinion of the appeal judge who rejected the views of a forensic scientist.

Are there any reports from the initial trial or the appeal in which the
"unreliability" is declared by a scientist (or statistician)? If not,
what are you going on? An erroneous decision by that judge?


of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

The coin-tossing example *in itself* is correct. Whether or not the
model applies to the DNA analysis of small samples is another question.

In order to answer that question one needs to understand the mechanisms
of DNA analysis. I understand that this takes place using a PCR
(Polymerase Chain Reaction) to clone any DNA into a large enough sample.
My wife is a biologist at the Sanger Institute and does these sorts of
things all the time in her research (cancer, malaria, etc).

If the PCR works and the DNA is not contaminated then you get enough DNA
to perform an analysis to compare with DNA from the suspect. This is
another separate process. I don't know if this can go wrong or not, but
I believe there are risks of false positives (the biased coin?)


I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data

Erm, presumably the test would be done on another small fragment of the
initial small sample on the knife. A PCR would be performed to amplify
the material so that it can be compared with the suspect yet again.

Naturally, they would not have used all of the sample in the first test
as the risk of it going wrong and the sample therefore being destroyed
is too high.

Since the DNA is cloned and therefore "amplified" one does not need a
lot of it to start with. A small piece of a small sample ought to suffice.

So, in this case the "biased coin" is rather small and has been chopped
up into pieces and tested, with each "piece of coin" behaving as though
it were the whole coin. (Unless the blood sample contains drops of blood
from more than one person?)

So, the several DNA analyses are indeed equivalent to new tosses of the
same coin, with the assessment of the coin being biased being equivalent
to the risk of false DNA positive matches?

Ergo, on the face of it, I think I believe Coralie Colmez.



(and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.

What is the unreliability other being subjectively opined by the appeal
judge?


Michael

tim...... posted
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data (and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.


The statistics of DNA analysis resembles Colmez' coin-tossing experiment
in some ways, but there are important differences. Whether the coin
analogy is appropriate or not depends on exactly what was wrong with the
scene-of-crime DNA sample. All we are told is that "the DNA sample was
tiny", but that isn't a useful statement.

The way it works is this: An individual's DNA fingerprint consists of a
certain number of parameters (loci) each of which can take several
different values (alleles). You might think of it a bit like someone's
name: for example my name might be expressed as
b i g l e s w a d e
where the allele at locus number 2 is 'i', the allele at locus 7 is 'w'
etc.

So if you find a scene-of-crime DNA sample that is "complete" - i.e. you
can read all of the alleles at all of the loci - you can compare it with
that of your suspect. If they all match, then it's not looking good for
him.

But the trouble with SOC-DNA samples is that they don't always contain a
record of all the alleles at all the loci - or if they do contain it, it
might be smothered by contamination from another source. The smaller the
amount of tissue present at the SOC, the more likely this is to happen.

The equivalent with my personal name analogy is where the police find
the corpse with a knife in his stomach, and just before he died he wrote
(in blood, on the wall next to him), "I was murdered by big**s**de",
where the asterisks are illegible splodges.

Clearly, I might be the murderer. But equally, the victim could have
been trying to name Big Al Spode, a well known local gangster. And it
doesn't matter how many times you examine this writing, you can't decide
whether the real killer is me or Al, simply because that information is
not present in the sample. It just isn't there, and you can't use
statistics to extract information that isn't there.

I'm not saying that this *is* what was wrong with the Kercher evidence,
just that it might be. And if it is, then Colmez' analogy is suspect.

And in fact it's more complicated than that, because certain DNA
analysis techniques can sometimes give you a probabilistic result for a
certain locus. E.g. in my analogy, that the missing fourth letter has a
90 per cent probability of being an "a", which would rule me out and put
Big Al in the frame.

However, such inferences (using what's called low copy number analysis)
are still very controversial, and are at the very least a great deal
more indirect than those from simple full-length DNA matching. Which
itself isn't perfect either, despite what they tell you.

--
Les

On Tuesday 30 April 2013 12:44 Big Les Wade wrote in uk.d-i-y:

tim...... posted
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data (and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.


The statistics of DNA analysis resembles Colmez' coin-tossing experiment
in some ways, but there are important differences. Whether the coin
analogy is appropriate or not depends on exactly what was wrong with the
scene-of-crime DNA sample. All we are told is that "the DNA sample was
tiny", but that isn't a useful statement.

The way it works is this: An individual's DNA fingerprint consists of a
certain number of parameters (loci) each of which can take several
different values (alleles). You might think of it a bit like someone's
name: for example my name might be expressed as
b i g l e s w a d e
where the allele at locus number 2 is 'i', the allele at locus 7 is 'w'
etc.

So if you find a scene-of-crime DNA sample that is "complete" - i.e. you
can read all of the alleles at all of the loci - you can compare it with
that of your suspect. If they all match, then it's not looking good for
him.

But the trouble with SOC-DNA samples is that they don't always contain a
record of all the alleles at all the loci - or if they do contain it, it
might be smothered by contamination from another source. The smaller the
amount of tissue present at the SOC, the more likely this is to happen.

The equivalent with my personal name analogy is where the police find
the corpse with a knife in his stomach, and just before he died he wrote
(in blood, on the wall next to him), "I was murdered by big**s**de",
where the asterisks are illegible splodges.

Clearly, I might be the murderer. But equally, the victim could have
been trying to name Big Al Spode, a well known local gangster. And it
doesn't matter how many times you examine this writing, you can't decide
whether the real killer is me or Al, simply because that information is
not present in the sample. It just isn't there, and you can't use
statistics to extract information that isn't there.

I'm not saying that this *is* what was wrong with the Kercher evidence,
just that it might be. And if it is, then Colmez' analogy is suspect.

And in fact it's more complicated than that, because certain DNA
analysis techniques can sometimes give you a probabilistic result for a
certain locus. E.g. in my analogy, that the missing fourth letter has a
90 per cent probability of being an "a", which would rule me out and put
Big Al in the frame.

However, such inferences (using what's called low copy number analysis)
are still very controversial, and are at the very least a great deal
more indirect than those from simple full-length DNA matching. Which
itself isn't perfect either, despite what they tell you.


Very nice and well explained example - thanks

--
Tim Watts Personal Blog: http://squiddy.blog.dionic.net/

http://www.sensorly.com/ Crowd mapping of 2G/3G/4G mobile signal coverage

Reading this on the web? See:
http://wiki.diyfaq.org.uk/index.php?title=Usenet

On 30/04/2013 12:44, Big Les Wade wrote:
However, such inferences (using what's called low copy number analysis)
are still very controversial, and are at the very least a great deal
more indirect than those from simple full-length DNA matching. Which
itself isn't perfect either, despite what they tell you.

I think the DNA case may be worse than your well described example.

With the splodges on the wall there are all sorts of non-destructive
analyses that can be carried out - photographs in UV light, oblique
lighting, image enhancement - none of which are destructive so you can
repeat them at will.

In the case of a tiny splodge on a knife you have to wipe it off in
order to test it, and once you've done your polymerase reactions you
don't have it any more.

Andy

"newshound" wrote in message
eb.com...
On 29/04/2013 21:24, tim...... wrote:
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data (and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.

Anyone any better ideas than me, what is the right answer here?

(I should really be posting this to ul, but I think the science experts
are in this group?)

TIA

tim

Well, presumably the first test was on a small sample taken from the
knife, and the proposed repeat test would have been on a different sample.
And that, I think, would improve the statistics although without knowing
the exact methodology it's difficult to be sure.

But it was the scientist suggested a repeat test and I am inclined to
think they would have understood the statistics.

Really

The history of such experts in court suggest that this faith is misplaced

tim

"Michael Kilpatrick" wrote in message
...
On 29/04/2013 21:24, tim...... wrote:
So according to:

http://www.bbc.co.uk/news/magazine-22310186

"Mathematician" Coralie Colmez claims that if you do a DNA analysis on
an unreliably small piece


Before we follow this line of thought, I observe from the BBC report that:

"the DNA sample was tiny, and the appeal judge thought the
evidence was unreliable, so he rejected a forensic scientist's
suggestion to have it tested again"

which suggests that the "unreliability" of the sample is the subjective
opinion of the appeal judge who rejected the views of a forensic
scientist.

Are there any reports from the initial trial or the appeal in which the
"unreliability" is declared by a scientist (or statistician)? If not, what
are you going on? An erroneous decision by that judge?

I'm accepting the position that the judge has primacy here.

The original judge may have been wrong, but I'm evaluating the additional
claim from the starting position that they were right.

of evidence and get the same result as the
first time, this strengthens the case that the original result is
correct and uses some coin-tossing explanation to "prove" her case.

The coin-tossing example *in itself* is correct. Whether or not the model
applies to the DNA analysis of small samples is another question.

In order to answer that question one needs to understand the mechanisms of
DNA analysis. I understand that this takes place using a PCR (Polymerase
Chain Reaction) to clone any DNA into a large enough sample. My wife is a
biologist at the Sanger Institute and does these sorts of things all the
time in her research (cancer, malaria, etc).

I agree with you, but the expert wasn't presenting a view on that. They
were presenting a strict statistical point, applicable to any scientific
test.

If the PCR works and the DNA is not contaminated then you get enough DNA
to perform an analysis to compare with DNA from the suspect. This is
another separate process. I don't know if this can go wrong or not, but I
believe there are risks of false positives (the biased coin?)


I disagree, her corroborating example repeats the same tests on
*different* data whereas doing the DNA test again would be repeating it
on the *same* data

Erm, presumably the test would be done on another small fragment of the
initial small sample on the knife. A PCR would be performed to amplify the
material so that it can be compared with the suspect yet again.

Naturally, they would not have used all of the sample in the first test as
the risk of it going wrong and the sample therefore being destroyed is too
high.

Since the DNA is cloned and therefore "amplified" one does not need a lot
of it to start with. A small piece of a small sample ought to suffice.

The judge in the first trial appears to have ruled that this is
amplification is unreliable. The experts view is that the unreliability of
a test is lessoned by doing the test twice. Personally I don't see it. You
are still testing the same sample and if you can only get a result by
amplification, and the judge has ruled that "amplification is unreliable, I
can't see that doing it twice (or three times) on that same sample, changes
that.

So, in this case the "biased coin" is rather small and has been chopped up
into pieces and tested, with each "piece of coin" behaving as though it
were the whole coin. (Unless the blood sample contains drops of blood from
more than one person?)

I can't see how that scales to a blood sample (on the basis that you can't
split up this coin as you are suggesting)

So, the several DNA analyses are indeed equivalent to new tosses of the
same coin, with the assessment of the coin being biased being equivalent
to the risk of false DNA positive matches?

OK, I still disagree



Ergo, on the face of it, I think I believe Coralie Colmez.



(and as such, is completely worthless extra
information).

I think that she has fundamentally missed the point about the
unreliability of the source data here.

What is the unreliability other being subjectively opined by the appeal
judge?

No idea

tim

On 30/04/2013 17:39, tim...... wrote:

The history of such experts in court suggest that this faith is misplaced


I suspect that you'll find that the vast majority of experts may be
correct. There is, however, a probability that they may be wrong (and
that's only when we hear about them in the news).

In the same way judges and juries may be.

Which is presumably why appeals and retrials, and multiple independent
expert witnesses are the order of the day!

Rather like tossing that biased coin multiple times, no?

Michael