Joi Ito and Iyad Rahwan on AI & Society

Beautiful day out

there. Thank you

for joining us here today.

This gives me great pleasure to

introduce you to

2 thought leaders

who actually inform

inspire and shape

my own thinking about

relationships between

technology and society

on at least a weekly basis.

And I'm not kidding

It's really fantastic to

have

for an hour and a bit

to talk about a big

topid: AI

and society.

Jot is an associate professor

at MIT media lab

where he leads the

scaleable corporations

group among other

things. He has done

really amazing work

over the last couple of

years looking at

the interplay between

autonomous systems

and society. How these

systems should interact

with eachother.

He recently published a

study in science that

got a lot of press

coverage.

addessing the question

whether we can program

moral principles

into autonomous vehicles.

and maybe he will talk a bit

more about that

and then of course

Joe

Director of MIT media lab

professor of practice.

Aperson who doesn't really

need an intro ,

so I'll keep it extremely brief.

Just by highlighting two

of the must reads

from recent months.

It's an interview he had,

a conversation actually with

President Obama

in the Wired Magazine.

on the future of the world,

addressing AI issues

among other topics.

And his book,

Whiplash, which is somehow a

survival guide for the

faster future that we're all

struggling with. I highly

recommend it as a reading

I greatly benefited fromi it.

So, these are not only 2

amazing thought leaders.

They're also wonderful collaborators

and colleagues

and I have the great privilege

through

Berkley and Kline team to work with b

bothof them as part of

our recently launched

joint venture.

The AI Ethics and Governance

Initiative. It's just wonderful

to have you here

and spend some time

with all of us

and share your thoughts

so thank you very much

and welcome.

(applause)

Thank you,

first of all

some of you may be here thinking

"wait, this isn't the talk

that I signed up for.

So to just give you some of

the prominence of this

originally I think there

was a book talk

that I was going to do

with Merckman and then

I said "oh, well, why don't

we bring somebody else interesting in

and Josh joined.

We were going to have a dialoge

about his book and

my book.

And he had a family emeregency

and coudln't make it.

I grabbed Jot and also

realized just as Ers was

saying, we're doing a lot

of work with the

Berkman Center on

AI and society and I thought

this would be a

sufficiently relevant topic to

what we were going to

talk about anyway

so it wouldn't be that

much false advertising.

and It was sort of an idea

that I think relates to my book as well.

One, I can't remember who it was

but a well known author told me

when you give

book talks, don't explain your

whole book because then no

one will have to buy it.

So this book actually started

about 4 years ago.

And we were just wrapping

it up as we saw a lot of

this AI, society,controversy/interests

start. So the book actually

sort of ends where our

exploration of AI and society begins.

So in a way, it overlaps

what the book is about

but is sufficently different

that you have to read the

book in order to

understand

the whole story

But, let me.

I'll just start a few remarks

We'll have .... present some of his work

and then we'll have a converstaion

with all of you

and feel free to interrupt

and ask questions

or disagree.

I think the(stammers)

I co-taught a class with Jonathan

in January in the winter semester.

His tradtional course he teaches

is called internet and society

it's a politics and technology

of control. .... was there

others were there, it was

a fun class.

But one of the sort of

framing pieces of how we

talked about this

was this sort of framing pieces of

how we talked about this.

Was this sort of lesigian?

picture that many of you

may have seen in

his book where

you have law at the top,

and then you have markets

on one side.

and you have norms on the other

and you have technology

underneath and you have you in

the middle

and some how

what you are able to do

is sort of determine by this

relationship between law. technology - I think technology

is on top and law is

down here.

But anyway,

somehow these all

effect eachother.

so you can create

technologies that effect

the law,

you can create laws that

effect norms,

youcan create norms that

effect technology

so some

realtionship between

norms, markets, law and technology

is how we need to be thinking

in order to

design all of these systems

so they work well in th

future. I think one

of the key reasons why the collaboration

between MIT and Harper Law School

Medialab and Berkman

so important is that

you kind of have to get all

of the pieces

and the people in the same room

because the problem is

once everyone has a solution

and they're trying to convince

eachother of the solution

it's, I call them, people

selling doll houses.

rather than legos .

What you want is a whole pile of legos

with lawyers and business people

and technologists and policy makers

playing with the legos

rather than

trying to sell eachother thier

own dollhouses.

That's what was sort

of fun with the class

is that I think

a lot of lawyers

realized that actually infact,

whether you're talking about bit

coin or differential privacy or AI

we still have a lot of choices

to make on the technology side

and in fact those can

be informed by

policy and law

and conversly, I think

a lot of the technologists thought

that law was something like

laws of physics that just are. But in

fact laws are the result

of lawyers and policy

makers taking to

technologists.

Imagining what society wants.

So we're sort of in the process right

now of

struggling through how

we think about this.

But Importantly,

it's already happening

so it's not like we

have that much time.

I think it was Pedro Demingo in

his book

says in master algorithm

and this isn't the exact

I'm paraphrasing the quote

it's something like

I'm less afraid of

a super intelligence coming

to take over the world

and more worried about

a stupid intelligence that's taken

over already.

You know?

I think that's very close to where we are.

I think if you see

Julie A's paper , article in

Propublic, I guess it was

a little over a year ago.

where she happens to find a district where

they're forced to disclose court records.

So she was specifically going after the

fact that machine learning

AI is now used by the

judiciary to set bail

to do parole

and even sentencing.

And they have this thing

called the risk score

where the machine sort

of pops up

after it does an assesment of

a person's history

looks at thier interviews and

she found, and this

is great, cause she's a

math matician

in a data sense. She crunched

all these numbers

and it shows

that for many cases for white people

is't sort of nearly random

in some cases.

So, it's a number but it's still almost random.

and then for black people

it's biased against them

and what's interesting

is when I talked to ...

a prosecuter the other day

he said well I love, they

love these numbers

because you get a risk score

that says okay,

this person has a

risk rating of 8

and so then the court can

say 'okay, we'll give you this bail

because the last thing that they

want is for them to give them

some bail

and then the person goes out and murders

somebody, it's sort of thier fault.

If they're taken the risk score

they can say, "I just looked at

the risk score

it absolves them of this

responsibility

and so there's this

really interesting question

that even at random

it's still, there' s this wierd moral

hazard that even though you

have agency, you're able to

push off

this responsibility to the machine, right?

And then you can sort of say,

well it was math.

and the problem right now

is these algorithms are running

on data sets and rating

systems that are closed.

We see this happening in a

variety of fields . I think we

see this happening

in the judiciary,

Which is a scary place for it

to be happening

And so part of this initiative with AI

fund that we're doing

we're going to try and look

at whether we can create

more transparency and auditability

we're also seeing it in medicine.

There's a study that I heard

that when a doctor overrulled the machine

in diagnostics the doctor was wrong

70 % of the time.

So what does that mean?

So if you're a doctor

and you know for a fact

that you're 70% likely

on average to be wrong

are you ever going to overrule

the machine?

And what about that 30%

where the doctors are right?

So, it creates a very difficult

situation.

You look at...

Imagine war

We talk about autonomous

weapons and there's this whole

fight about

it, but what if all of the data

and not what if,

In fact, all of the data

that's driving intellegence

the way that you get on to

the termination list

as a target,

a lot of it involves statistical

analysis of your activity

your emotions, your calls

and there's this great interview

I think it was in the Independent or

it was Indpendent.

There was this guy who

I think he was in

Pakistan

I'm gonna get this wrong

I'll um, but it's close.

But he had been attacked

a number of times

where the collateral damage

was family members

being dead so he knew he

was on the kill list

but he didn't know how to get off

so he goes to London

to kind of fight for

'wait, look at me, talk to me

I'm on this kill list

but I'm not a bad guy.

Somehow

you got the wrong person.

But there's no interface

in which he can sort of

lobby and petition for getting off

this kill list.

So even though the person

controlling the drone strike and

pushing the button

maybe a human being,

If all of the data that's feeding into, or

a substantial amount of data that's

feeding into the decision to

put the person on

the kill list

is from a machine,

I don't know how that's that

different

from the machine actually being charged.

So we talk about sort of these

future autonomous systems

and robots running around

and killing people as

a sort of scary thing.

But if we are just pushing a button that the

robot just tells us to do

ABC or D but robot

says it's C, you're going to push C.

Apparently that was how Kisenger controlled Nixon

was through his elbow.

The anwer was always C.

But anyway,

the.... that actually is

when we think about practice.

We may already be in autonomous

mode in many things.

And then I'm going to T up to

Y? which is one of the

first places where the rubber meets

the road is with autonomous vehicles.

and a lot of the people that

I talk to say

that while the real soul searching

around this is going to happen

when the next big autonmous vehicle

accident happens where

it's clearly the machine's fault,

How is that going to play out?

So that may be one of the things

But the last thing that

I'll say is that I think

this is where the media lab

is excited. I think it's kind of

an interface design problem

because part of what the

problem is is that you may think

that by pushing , the button, the right to

overrule the computer the right to

launch the missle

may be your finger

if you have no choice,

morally or statistically other than

to push the button

you're not in charge anymore. Right?

So what I think we need to think about

is how to we bring

society and humans into the decision making

process so that the answer that

we derive involves human beings

and how does that interface hapen? what is the

right way to do it

Because I think what we are going to end

up with is collective decision making.

machines. and what we want to

not be in is human agency with no real decision

making ability.

And then we can talk more about some

of the ideas.

but I'll hand it over to Jot,

Thank you.

So I'll just give a short

overview of

the research we' ve been doing

on autonomous vehicles

I'm not a driverless car expert.

I don't build driverless cars.

But I'm interested in them as

as kind of a social phenomenon

and the reason has to do with this dilema that

Steve will keep discussing.

You know, what if it's an

autonomous car

that is going to for some

reason harm a bunch of

pedestrians

crossing the street because the

brakes are broken

or because they jumped in front

of it or whatever.

But the car can swerve and kill one

bystander on the other side.

in order to minimize harm

in order to save 5 or 10 people

should the car do this?

And who should decide?

And more interestingly,

what if the car could

swerve and hit a wall harming

the passenger or killing the passenger.?

In order to save these people?

Should the car do this as well?

What does the car

have a duty towards.?

Minimizing harm?utilitarian principle?

Protection of the owner or passengers in the car?

Duty toward them? Or something else? A sort of negotiation

inbetween?

Do we ignore this problem?

do we just say

well let the car deal with this problem

and it seems to be a very

controversial topic because

there are lots of people

who love this,

and lots of people who hate this.

and people who hate this say,

Well this is never going to happen

it's just so statistically unlikely

and I think that kind

of misses the point

because this is a invitro e xploration

of a principle so you strip

away all of the things that

don't matter in the real world

so you can isolate the factor

you know, does drug x

cause this particular

reation in a cell for example?

You know, you don't do this in the

forest you do it in a petridish.

And this is the petri dish for studying

human perception of machine ethics

and what other factors do people

seem to be ticked off by?