Do we see reality as it is?

I love a great mystery,

and I'm fascinated by the greatest
unsolved mystery in science,

perhaps because it's personal.

It's about who we are,

and I can't help but be curious.

The mystery is this:

what is the relationship
between your brain

and your conscious experiences,

such as your experience
of the taste of chocolate

or the feeling of velvet?

Now, this mystery is not new.

In 1868, Thomas Huxley wrote,

"How it is that anything so remarkable
as a state of consciousness comes about

as the result of irritating nervous tissue

is just as unaccountable

as the appearance of the Genie
when Aladdin rubbed his lamp.

Now, Huxley knew that brain activity

and conscious experiences are correlated,

but he didn't know why.

To the science of his day,
it was a mystery.

In the years since Huxley,

science has learned a lot
about brain activity,

but the relationship
between brain activity

and conscious experiences
is still a mystery.

Why? Why have we made so little progress?

Well, some experts think
that we can't solve this problem

because we lack the necessary concepts

and intelligence.

We don't expect monkeys to solve
problems in quantum mechanics,

and as it happens, we can't expect
our species to solve this problem either.

Well, I disagree. I'm more optimistic.

I think we've simply
made a false assumption.

Once we fix it, we just
might solve this problem.

Today, I'd like tell you
what that assumption is,

why it's false, and how to fix it.

Let's begin with a question:

do we see reality as it is?

I open my eyes

and I have an experience that I describe

as a red tomato a meter away.

As a result, I come to believe
that in reality,

there's a red tomato a meter away.

I then close my eyes, and my experience
changes to a grey field,

but is it still the case that in reality,
there's a red tomato a meter away?

I think so, but could I be wrong?

Could I be misinterpreting
the nature of my perceptions?

We have misinterpreted
our perceptions before.

We used to think the Earth is flat,

because it looks that way.

Pythagorus discovered that we were wrong.

Then we thought that the Earth

is the unmoving center of the Universe,

again because it looks that way.

Copernicus and Galileo discovered,
again, that we were wrong.

Galileo then wondered if we might
be misinterpreting our experiences

in other ways.

He wrote: "I think that tastes,
odors, colors, and so on

reside in consciousness.

Hence if the living creature were removed,

all these qualities would be annihilated.

Now, that's a stunning claim.

Could Galileo be right?

Could we really be misinterpreting
our experiences that badly?

What does modern science
have to say about this?

Well, neuroscientists tell us
that about a third of the brain's cortex

is engaged in vision.

When you simply open your eyes
and look about this room,

billions of neurons
and trillions of synapses are engaged.

Now, this is a bit surprising,

because to the extent
we think about vision at all,

we think of it as like a camera.

It just takes a picture
of objective reality as it is.

Now, there is a part of vision
that works like a camera:

the eye has a lens that focuses an image
on the back of the eye

where there are 130 million
photoreceptors,

so the eye is like a 130 megapixel camera.

But that doesn't explain
the billions of neurons

and trillions of synapses
that are engaged in vision.

What are these neurons up to?

Well, neuroscientists tell us
that they are creating, in real time,

all the shapes, objects, colors,
and motions that we see.

It feels like we're just taking a snapshot
of this room the way it is,

but in fact, we're constructing
everything that we see.

We don't construct
the whole world at once.

We construct what we need in the moment.

Now, there are many demonstrations
that are quite compelling

that we construct what we see.

I'll just show you two.

In this example, you see some red discs

with bits cut out of them,

but if I just rotate
the disks a little bit,

suddenly, you see a 3D cube
pop out of the screen.

Now, the screen of course is flat,

so the three dimensional cube
that you're experiencing

must be your construction.

In this next example,

you see glowing blue bars

with pretty sharp edges

moving across a field of dots.

In fact, no dots move.

All I'm doing from frame to frame

is changing the colors of dots

from blue to black or black to blue.

But when I do this quickly,

your visual system creates
the glowing blue bars

with the sharp edges and the motion.

There are many more examples,
but these are just two

that you construct what you see.

But neuroscientists go further.

They say that we reconstruct reality.

So, when I have an experience
that I describe as a red tomato,

that experience is actually
an accurate reconstruction

of the properties of a real red tomato

that would exist
even if I weren't looking.

Now, why would neuroscientists
say that we don't just construct,

we reconstruct?

Well, the standard argument given

is usually an evolutionary one.

Those of our ancestors
who saw more accurately

had a competitive advantage compared
to those who saw less accurately,

and therefore they were more likely
to pass on their genes.

We are the offspring of those
who saw more accurately,

and so we can be confident that,
in the normal case,

our perceptions are accurate.

You see this in the standard textbooks.

One textbook says, for example:

"Evolutionarily speaking,

vision is useful precisely
because it is so accurate."

So the idea is that accurate perceptions

are fitter perceptions.

They give you a survival advantage.

Now, is this correct?

Is this the right interpretation
of evolutionary theory?

Well, let's first look at a couple
of examples in nature.

The Australian jewel beetle

is dimpled, glossy, and brown.

The female is flightless.

The male flies, looking, of course,
for a hot female.

When he finds one, he alights, and mates.

There's another species in the outback,

homo sapiens.

The male of this species
has a massive brain

that it uses to hunt for cold beer.

(Laughter)

And when he finds one, he drains it,

and sometimes throws the bottle
into the outback.

Now, as it happens, these bottles
are dimpled, glossy,

and just the right shade of brown

to tickle the fancy of these beetles.

The males swarm all over the bottles

trying to mate.

They lose all interest
in the real females.

Classic case of the male
leaving the female for the bottle.

(Laughter) (Applause)

The species almost went extinct.

Australia had to change its bottles

to save its beetles.

Now, the males had successfully
found females for thousands,

perhaps millions of years.

It look like they saw reality as it is,

but apparently not: evolution
had given them a hack.

A female is anything dimpled,
glossy, and brown,

the bigger the better.

(Laughter)

Even while crawling all over the bottle,
the male couldn't discover his mistake.

Now, you might say, beetles, sure,
they're very simple creatures,

but surely not mammals.

Mammals don't rely on tricks.

Well, I won't dwell on this,
but you get the idea. (Laughter)

So this raises an important
technical question:

does natural selection really favor
seeing reality as it is?

Fortunately, we don't have
to wave our hands and guess:

evolution is a mathematically
precise theory.

We can use the equations of evolution
to check this out.

We can have various organisms
in artificial worlds compete

and see which survive and which thrive,

which sensory systems are more fit.

A key notion in those
equations is fitness.

Consider this steak:

what does this steak do
for the fitness of an animal?

Well, for a hungry lion looking to eat,

it enhances fitness.

For a well-fed lion looking to mate,
it doesn't enhance fitness.

And for a rabbit in any state,
it doesn't enhance fitness,

so fitness does depend
on reality as it is, yes,

but also the organism,
its state, and its action.

Fitness is not the same thing
as reality as it is,

and it's fitness,
and not reality as it is,

that figures centrally
in the equations of evolution.

So, in my lab,

we have run hundreds of thousands
of evolutionary game simulations

with lots of different
randomly chosen worlds

and organisms that compete
for resources in those worlds.

Some of the organisms
see all of the reality,

others see just part of the reality,

and some see none of the reality,

only fitness.

Who wins?

Well, I hate to break it to you,

but perception of reality goes extinct.

In almost every simulation,

organisms that see none of reality

but are just tuned to fitness

drive to extinction all the organisms
that perceive reality as it is.

So the bottom line is, evolution
does not favor vertical,

or accurate perceptions.

Those perceptions of reality go extinct.

Now, this is a bit stunning:

how can it be that not seeing
the world accurately

gives us a survival advantage?

That is a bit counterintuitive.

But remember the jewel beetle.

The jewel beetle survived
for thousands, perhaps millions of years,

using simple tricks and hacks.

What the equations
of evolution are telling us

is that all organisms, including us,
are in the same boat as the jewel beetle.

We do not see reality as it is:

we're shaped with tricks
and hacks that keep us alive.

Still,

we need some help with our intuitions.

How can not perceiving
reality as it is be useful?

Well, fortunately, we have
a very helpful metaphor:

the desktop interface on your computer.