Should you trust unanimous decisions? - Derek Abbott

Imagine a police line-up
where ten witnesses

are asked to identify a bank robber
they glimpsed fleeing the crime scene.

If six of them pick out the same person,

there's a good chance
that's the real culprit,

and if all ten make the same choice,

you might think the case is rock solid,

but you'd be wrong.

For most of us,
this sounds pretty strange.

After all, much of our society
relies on majority vote and consensus,

whether its politics,

business,

or entertainment.

So it's natural to think
that more consensus is a good thing.

And up until a certain point,
it usually is.

But sometimes, the closer you start to get
to total agreement,

the less reliable the result becomes.

This is called the paradox of unanimity.

The key to understanding
this apparent paradox

is in considering the overall level
of uncertainty

involved in the type of situation
you're dealing with.

If we asked witnesses to identify
the apple in this line up, for example,

we shouldn't be surprised
by a unanimous verdict.

But in cases where we have
reason to expect some natural variance,

we should also expect varied distribution.

If you toss a coin one hundred times,

you would expect to get heads
somewhere around 50% of the time.

But if your results started
to approach 100% heads,

you'd suspect that something was wrong,

not with your individual flips,

but with the coin itself.

Of course, suspect identifications aren't
as random as coin tosses,

but they're not as clear cut
as telling apples from bananas, either.

In fact, a 1994 study found
that up to 48% of witnesses

tend to pick the wrong
person out of a line-up,

even when many
are confident in their choice.

Memory based on short glimpses
can be unreliable,

and we often overestimate
our own accuracy.

Knowing all this,

a unanimous identification starts to seem
less like certain guilt,

and more like a systemic error,

or bias in the line-up.

And systemic errors don't just appear
in matters of human judgement.

From 1993-2008,

the same female DNA was found
in multiple crime scenes around Europe,

incriminating an elusive killer
dubbed The Phantom of Heilbronn.

But the DNA evidence was so consistent
precisely because it was wrong.

It turned out that the cotton swabs
used to collect the DNA samples

had all been accidentally contaminated
by a woman working in the swab factory.

In other cases, systematic errors arise
through deliberate fraud,

like the presidential referendum held
by Saddam Hussein in 2002,

which claimed a turnout of 100% of voters
with all 100% supposedly voting in favor

of another seven-year term.

When you look at it this way,

the paradox of unanimity isn't actually
all that paradoxical.

Unanimous agreement
is still theoretically ideal,

especially in cases when you'd expect very
low odds of variability and uncertainty,

but in practice,

achieving it in situations where
perfect agreement is highly unlikely

should tell us that there's probably
some hidden factor affecting the system.

Although we may strive for harmony
and consensus,

in many situations, error and disagreement
should be naturally expected.

And if a perfect result seems too good
to be true,

it probably is.