-
Not Synced
I work with a bunch of mathematicians, philosophers,
and computer scientists
-
Not Synced
and we sit around and think about the future
of machine intelligence,
-
Not Synced
among other things.
-
Not Synced
Some people think that some of
these people are
-
Not Synced
science fiction-y,
-
Not Synced
far out there, crazy.
-
Not Synced
But I like to say,
-
Not Synced
"Okay, let's look at the modern
human condition."
-
Not Synced
(Laughter)
-
Not Synced
This is the normal way for things to be.
-
Not Synced
But, if we think about it,
-
Not Synced
we are actually recently arrived
guests on this planet.
-
Not Synced
The human species --
-
Not Synced
think of it the world was created
one year ago,
-
Not Synced
the human species, then, would be
10 minutes old.
-
Not Synced
The industrial era started two seconds ago.
-
Not Synced
Another way to think of this,
-
Not Synced
if you think of world GDP
over the last 10,000 years,
-
Not Synced
I've actually taken the trouble
to plot this for you in a graph.
-
Not Synced
It looks like this.
-
Not Synced
(Laughter)
-
Not Synced
It's a curious shape for a normal condition.
-
Not Synced
I sure wouldn't want to sit on it.
-
Not Synced
(Laughter)
-
Not Synced
Let's ask ourselves,
-
Not Synced
what is the cost of this current anomaly?
-
Not Synced
Some people would say it's technology.
-
Not Synced
It's true.
-
Not Synced
Technology has accumulated through
human history,
-
Not Synced
and right now, technology advances
extremely rapidly,
-
Not Synced
that is the proximate cause,
-
Not Synced
that's why we are currently so very productive.
-
Not Synced
But I like to think back further to the ultimate cause.
-
Not Synced
Look at these two highly distinguished gentlemen.
-
Not Synced
We have Kanzi,
-
Not Synced
he's mastered 200 lexical tokens,
-
Not Synced
an incredible feat.
-
Not Synced
And Ed Witten unleashed the second
string theory revolution.
-
Not Synced
If we look under the hood, this is
what we find:
-
Not Synced
basically the same thing.
-
Not Synced
One is a little larger,
-
Not Synced
it maybe also has a few tricks
in the way it's wired.
-
Not Synced
These invisible differences cannot
be too complicated
-
Not Synced
because there've only been 250,000
generations since
-
Not Synced
our last common ancestor.
-
Not Synced
We know that complicated mechanisms
that a long time to evolve.
-
Not Synced
So a bunch of relatively minor changes
-
Not Synced
from Kanzi to Witten.
-
Not Synced
From broken-off tree branches,
-
Not Synced
to intercontinental balistic missles.
-
Not Synced
So this then seems pretty obvious
that
-
Not Synced
everything we've achieved, pretty much,
-
Not Synced
and everything we care about
depends crucially
-
Not Synced
on relatively minor changes that made
-
Not Synced
the human mind.
-
Not Synced
And the collaraly, of course, is that
-
Not Synced
any further changes that could
significantly change
-
Not Synced
the substrate of thinking
-
Not Synced
could have potentially enormous consequences.
-
Not Synced
Some of my colleagues think we're
on the verge
-
Not Synced
of something that could cause profound change
-
Not Synced
in the substrate.
-
Not Synced
That is, machine super intelligence.
-
Not Synced
Artificual intelligence used to be
-
Not Synced
putting commands in a box.
-
Not Synced
You have human programmers that would
-
Not Synced
painstakingly hand-craft items.
-
Not Synced
You build up these expert systems,
-
Not Synced
and they were kind of useful
for some purposes,
-
Not Synced
But they were very brittle,
-
Not Synced
you couldn't scale them.
-
Not Synced
Basically, you got out only
what you put in.
-
Not Synced
But since then, a paradigm shift has
taken place
-
Not Synced
in the field of artificial intelligence.
-
Not Synced
Today, the action is really around machine learning.
-
Not Synced
So rather than handcrafting knowledge
representations and features,
-
Not Synced
we create algorithms that learn,
-
Not Synced
often from raw perceptual data.
-
Not Synced
Basically the same thing
that the human infant does.
-
Not Synced
The result is AI that is not limited
to one domain,
-
Not Synced
the same system can learn to translate
between any pairs of languages,
-
Not Synced
or learn to play any computer game
at the Atari console.
-
Not Synced
Now of course, AI is still no where near having
the same powerful, cross-domain
-
Not Synced
ability to learn and plan as a human being has,
-
Not Synced
the cortex still has some algorithmic tricks
-
Not Synced
that we don't yet know
how to match in machines.
-
Not Synced
But so the question is,
-
Not Synced
how far are we from being able
to match those tricks?
-
Not Synced
A couple of years ago, we did a survey
-
Not Synced
of some of the world's leading
AI experts
-
Not Synced
to see what they think
-
Not Synced
and one of the questions we asked was,
-
Not Synced
"By which year do you think
there's a 50 percent probaility
-
Not Synced
that we will have achieved
human-level machine intelligence?"
-
Not Synced
We defined human-level here as the ability
-
Not Synced
to perform any job at least as well
as an adult human level,
-
Not Synced
not just within some limited domain.
-
Not Synced
And the median answer was 2040 or 2050,
-
Not Synced
depending on precisely which
group of experts we ask.
-
Not Synced
Now, it could happen much, much later,
-
Not Synced
or sooner, the truth is
nobody really knows.
-
Not Synced
What we do know is that
the ultimate limit
-
Not Synced
to information processing in machine substrate,
-
Not Synced
lie far outside the limits of biological tissue.
-
Not Synced
This comes down to physics.
-
Not Synced
A biological neuron fires, maybe,
at 200 Hertz, 200 times a second.
-
Not Synced
But even a present-day transistor
operates at a giga-hert.
-
Not Synced
Neurons propagate slowly in axons,
-
Not Synced
100 meters per second, tops.
-
Not Synced
But in computers, signals can travel
at the speed of light.
-
Not Synced
There's also size limitations,
-
Not Synced
a human brain has to fit inside a cranium,
-
Not Synced
but a computer can be the size
of a warehouse or larger.
-
Not Synced
So the potential of super intelligence
lies dormant in matter,
-
Not Synced
much like the power of the atom lay dormant
throughout human history,
-
Not Synced
patiently waiting there until 1945.
-
Not Synced
In this century, scientists may learn
to awake the power of artificial intelligence.
-
Not Synced
And I think we might then see
an intelligence explosion.
-
Not Synced
Now most people,
-
Not Synced
when they think about what is smart
and what is dumb,
-
Not Synced
I think I have in mind a picture
roughly like this.
-
Not Synced
So at one hand, we have the village idiot,
-
Not Synced
and then far over at the other side,
-
Not Synced
we have Ed Witten,
-
Not Synced
or Albert Einsten or whoever your favorite
guru is.
-
Not Synced
But I think from the point of view
of Artificial Intelligence,
-
Not Synced
the true picture is actually probably
more like this:
-
Not Synced
AI starts out at this point here,
-
Not Synced
at zero intelligence,
-
Not Synced
and then, after many, many
years of really hard work,
-
Not Synced
eventually we get to mouse-level
artificial intelligence,
-
Not Synced
something that can navigate
cluttered environments
-
Not Synced
as well as a mouse can.
-
Not Synced
And then, after many, many more years
of really hard work, lots of investment,
-
Not Synced
maybe eventually we get to
chimpanzee-level artificial intelligence.
-
Not Synced
And then, after even more years
of really, really hard work,
-
Not Synced
we get village idiot artificial intelligence.
-
Not Synced
And a few moments later,
we are beyond Ed Witten.
-
Not Synced
The train doesn't stop at
humanville station.
-
Not Synced
It's likely, rather, to swoosh right by.
-
Not Synced
Now this has profound implications,
-
Not Synced
particularly when it comes
to questions of power.
-
Not Synced
For example, chimpanzees are strong,
-
Not Synced
pound for pound, a chimpanzee is about
twice as strong as a fit human male.
-
Not Synced
And yet, the fate of Kanzi and his pals
depends a lot more
-
Not Synced
on what we humans do than on
what the chimpanzees do themselves.
-
Not Synced
Once there is super intelligence,
-
Not Synced
the fate of humanity may depend
on what the super intelligence does.
-
Not Synced
Think about it: machine intelligence
is the last invention
-
Not Synced
that humanity will ever need to make.
-
Not Synced
Machines will then be better
at inventing than we are,
-
Not Synced
and they'll be doing so on digital timescales.
-
Not Synced
What this means is basically
a telescoping of the future.
-
Not Synced
Think of all the crazy technologies that you
could have imaged
-
Not Synced
that humans could have developed
in the fullness of time:
-
Not Synced
cures for aging, space colonization,
-
Not Synced
self-replicating nanobots
-
Not Synced
or uploading of minds into computers,
-
Not Synced
all kinds of science fiction-y stuff
-
Not Synced
that's nevertheless consistent
with the laws of physics.
-
Not Synced
All of this, super intelligence
could develop
-
Not Synced
and possibly, quite rapidly.
-
Not Synced
Now, super intelligence with such
technological maturity
-
Not Synced
would be extremely powerful,
-
Not Synced
and at least in some scenarios,
-
Not Synced
it would be able to get
what it wants.
-
Not Synced
We would then have a future
that would be shaped
-
Not Synced
by the preferences of this AI.
-
Not Synced
Now a good question is, what are
those preferences?
-
Not Synced
Here it gets trickier.
-
Not Synced
To make any headway with this,
-
Not Synced
we must first, first of all,
-
Not Synced
avoid anthropomorphizing.
-
Not Synced
this is ironic because
every newspaper article
-
Not Synced
about the future of AI has a picture of this:
-
Not Synced
So I think what we need to do is
to conceive of the issue
-
Not Synced
more abstractly,
-
Not Synced
not in terms of vivid Hollywood scenarios.
-
Not Synced
We need to think of intelligence as an
optimization process,
-
Not Synced
a process that steers the future
into a particular set of configurations.
-
Not Synced
As super intelligence --
-
Not Synced
it's a really strong optimization process.
-
Not Synced
It's extremely good at using
available means
-
Not Synced
to achieve a state in which its
goal is realized.
-
Not Synced
This means that there is no necessary
conenction between
-
Not Synced
being highly intelligent in this sense,
-
Not Synced
and having an objective that we humans
-
Not Synced
would find worthwhile or meaningful.
-
Not Synced
Suppose we give AI the goal
to make humans smile.
-
Not Synced
When the AI is weak, it performs useful
or amusing actions
-
Not Synced
that cause its user to smile.
-
Not Synced
When the AI becomes super intelligent,
-
Not Synced
it realizes that there is
a more effective way
-
Not Synced
to achieve this goal:
-
Not Synced
take control of the world
-
Not Synced
and stick electrodes into
the facial muscles of humans
-
Not Synced
to cause constant, beaming grins.
-
Not Synced
Another example, suppose we give AI
the goal to solve
-
Not Synced
a difficult mathematical problem.
-
Not Synced
When the AI becomes super intelligent,
-
Not Synced
it realizes that the most effective way
to get the solution to this problem
-
Not Synced
is to by transforming the planet
into a giant computer,
-
Not Synced
so as to increase its thinking capacity.
-
Not Synced
And notice that this gives the AIs
an instrumental reason
-
Not Synced
to do things to us that we
might not approve us.
-
Not Synced
Human beings in this model are threats,
-
Not Synced
we could prevent the mathematical problem
from being solved.
-
Not Synced
Of course, perceivably things won't
go wrong in these particular ways,
-
Not Synced
these are cartoon examples.
-
Not Synced
But the general point here is important:
-
Not Synced
if you create a really powerful
optimization process
-
Not Synced
to maximize for objective x,
-
Not Synced
you better make sure that
your definition of x
-
Not Synced
incorporates everything you care about.
-
Not Synced
This is a lesson that's also taught
in many a myth.
-
Not Synced
Kind Midas wishes that everything
he touches be turned into gold.
-
Not Synced
He touches his daughter,
she turns into fold.
-
Not Synced
He touches his food, it turns into gold.
-
Not Synced
This could become practically relevant,
-
Not Synced
not just for a metaphor for greed,
-
Not Synced
but an illustration of what happens
if you create
-
Not Synced
a powerful optimization process
-
Not Synced
and give it misconceived
or poorly specified goals.
-
Not Synced
Now you might say, "If a computer starts
sticking electrodes into people's faces,
-
Not Synced
we'd just shut it off."
-
Not Synced
A: This is not necessarily so easy to do
if we've grown dependent
-
Not Synced
on the system,
-
Not Synced
like where is the off switch to the internet?
-
Not Synced
B: Why haven't the chimpanzees
flicked the off-switch to humanity,
-
Not Synced
or the neandrathals?
-
Not Synced
They certainly had reasons.
-
Not Synced
We have an off switch, for example,
right here.
-
Not Synced
[choking sound]
-
Not Synced
The reason is that we are
an intelligent adversary,
-
Not Synced
we can anticipate threats
and we can plan around them.
-
Not Synced
But so could a super intelligent agent,
-
Not Synced
and it would be much better at that
than we are.
-
Not Synced
The point is, we should not be confident
that we have this under control here.
-
Not Synced
And we could try to make our job
a little bit easier by, say,
-
Not Synced
putting the AI in a box,
-
Not Synced
like a secure software environment,
-
Not Synced
a virtual reality simulation
from which it cannot escape.
-
Not Synced
But how confident can we be that
the AI couldn't find a bug,
-
Not Synced
given that even human hackers find bugs
all the time,
-
Not Synced
I'd say, probably not very confident.
-
Not Synced
So we disconnect the ethernet cable
to create an air gap,
-
Not Synced
but again, like nearly human hackers
routinely transgress air gaps
-
Not Synced
using social engineering.
-
Not Synced
Like right now as I speak,
-
Not Synced
I"m sure there is some employee
out there somewhere
-
Not Synced
who's been talked into handing out
her account details
-
Not Synced
by some by somebody claiming
to be from the IT department.
-
Not Synced
More creative scenarios are also possible,
-
Not Synced
like if you're the AI, you can imagine
wiggling electroces around
-
Not Synced
in your internal circuitry to create
radio waves that you can use to communicate.
-
Not Synced
Or maybe you could pretend to malfunction,
-
Not Synced
and then when the programmers open
you up to see what went wrong with you,
-
Not Synced
they look at the source code -- BAM! --
-
Not Synced
the manipulation can take place.
-
Not Synced
Or it could output the blueprint
to a really nifty technology
-
Not Synced
and when we implement it,
-
Not Synced
it has some surreptitious side effect
that the AI had planned.
-
Not Synced
The point here is that we should
not be confident in our ability
-
Not Synced
to keep a super intelligent genie
locked up in its bottle forever.
-
Not Synced
Sooner or later, it will out.
-
Not Synced
I believe that the answer here
is to figure out
-
Not Synced
how to create super intelligent AI
such that even if,
-
Not Synced
when it escapes,
-
Not Synced
it is still safe because it
is fundamentally on our side
-
Not Synced
because it shares our values.
-
Not Synced
I see no way around
this difficult problem.
-
Not Synced
Now, I'm actually fairly optimistic
that this problem can be solved.
-
Not Synced
We wouldn't have to write down
a long list of everything we care aobut
-
Not Synced
or worse yet, spell it out
in some computer language
-
Not Synced
like C ++ or Python,
-
Not Synced
that would be a task beyond hopeless.
-
Not Synced
Instead, we would create an AI
that uses its intelligence
-
Not Synced
to learn what we value
-
Not Synced
and its motivation system is constructed
in such a way that it is motivated
-
Not Synced
to pursue our values or to perform actions
that it predicts we would approve of.
-
Not Synced
We would thus leverage its intelligence
as much as possible
-
Not Synced
to solve the problem of value loading.
-
Not Synced
This can happen,
-
Not Synced
and the outcome could be
very good for humanity.
-
Not Synced
But it doesn't happen automatically.
-
Not Synced
The initial conditions
for the intelligent explosion
-
Not Synced
might need to be set up
in just the right way
-
Not Synced
if we are to have a controlled detonation.
-
Not Synced
The values that the AI has need
to match ours,
-
Not Synced
not just in the familiar context,
-
Not Synced
like where we can easily check
how the AI behaves,
-
Not Synced
but also in all novel contexts
that the AI might encounter
-
Not Synced
in the indefinite future.
-
Not Synced
And there are also some esoteric issues
that would need to be solved,
-
Not Synced
sorted out,
-
Not Synced
the exact decisions of its decision theory,
-
Not Synced
how to deal with logical uncertainty,
and so forth.
-
Not Synced
So the technical problems that need
to be solved to make this work
-
Not Synced
look quite difficult,
-
Not Synced
-- not as difficult as making
a super intelligent AI,
-
Not Synced
but fairly difficult.
-
Not Synced
Here is the worry:
-
Not Synced
making super intelligent AI
is a super hard challenge.
-
Not Synced
Making super intelligent AI
that is safe
-
Not Synced
involves some additional challenge
on top of that.
-
Not Synced
The risk is that if somebody
figures out how to crack
-
Not Synced
the first challenge without also
having cracked
-
Not Synced
the additional challenge
of ensuring perfect safety.
-
Not Synced
So I think that we should
work out a solution
-
Not Synced
to the controlled problem in advance,
-
Not Synced
so that we have it available
by the time it is needed.
-
Not Synced
Now it might be that we cannot
solve the entire controlled problem
-
Not Synced
in advance because maybe some element
can only be put in place
-
Not Synced
once you know the details of
the architecture
-
Not Synced
where it will be implemented.
-
Not Synced
But the more of the controlled problem
that we solve in advance,
-
Not Synced
the better the odds that the transition
to the machine intelligence era
-
Not Synced
will go well.
-
Not Synced
This to me looks like a thing
that is well worth doing
-
Not Synced
and I can imagine that if
things turn out okay,
-
Not Synced
that people in a million years
from now
-
Not Synced
look back at this century
-
Not Synced
and it might well be that
they say
-
Not Synced
that he one thing we did
that really mattered
-
Not Synced
was to get this thing right.
-
Not Synced
Thank you.
-
Not Synced
(Applause)