WEBVTT 99:59:59.999 --> 99:59:59.999 Beautiful day out 99:59:59.999 --> 99:59:59.999 there. Thank you 99:59:59.999 --> 99:59:59.999 for joining us here today. 99:59:59.999 --> 99:59:59.999 This gives me great pleasure to 99:59:59.999 --> 99:59:59.999 introduce you to 99:59:59.999 --> 99:59:59.999 2 thought leaders 99:59:59.999 --> 99:59:59.999 who actually inform 99:59:59.999 --> 99:59:59.999 inspire and shape 99:59:59.999 --> 99:59:59.999 my own thinking about 99:59:59.999 --> 99:59:59.999 relationships between 99:59:59.999 --> 99:59:59.999 technology and society 99:59:59.999 --> 99:59:59.999 on at least a weekly basis. 99:59:59.999 --> 99:59:59.999 And I'm not kidding 99:59:59.999 --> 99:59:59.999 It's really fantastic to 99:59:59.999 --> 99:59:59.999 have 99:59:59.999 --> 99:59:59.999 for an hour and a bit 99:59:59.999 --> 99:59:59.999 to talk about a big 99:59:59.999 --> 99:59:59.999 topid: AI 99:59:59.999 --> 99:59:59.999 and society. 99:59:59.999 --> 99:59:59.999 Jot is an associate professor 99:59:59.999 --> 99:59:59.999 at MIT media lab 99:59:59.999 --> 99:59:59.999 where he leads the 99:59:59.999 --> 99:59:59.999 scaleable corporations 99:59:59.999 --> 99:59:59.999 group among other 99:59:59.999 --> 99:59:59.999 things. He has done 99:59:59.999 --> 99:59:59.999 really amazing work 99:59:59.999 --> 99:59:59.999 over the last couple of 99:59:59.999 --> 99:59:59.999 years looking at 99:59:59.999 --> 99:59:59.999 the interplay between 99:59:59.999 --> 99:59:59.999 autonomous systems 99:59:59.999 --> 99:59:59.999 and society. How these 99:59:59.999 --> 99:59:59.999 systems should interact 99:59:59.999 --> 99:59:59.999 with eachother. 99:59:59.999 --> 99:59:59.999 He recently published a 99:59:59.999 --> 99:59:59.999 study in science that 99:59:59.999 --> 99:59:59.999 got a lot of press 99:59:59.999 --> 99:59:59.999 coverage. 99:59:59.999 --> 99:59:59.999 addessing the question 99:59:59.999 --> 99:59:59.999 whether we can program 99:59:59.999 --> 99:59:59.999 moral principles 99:59:59.999 --> 99:59:59.999 into autonomous vehicles. 99:59:59.999 --> 99:59:59.999 and maybe he will talk a bit 99:59:59.999 --> 99:59:59.999 more about that 99:59:59.999 --> 99:59:59.999 and then of course 99:59:59.999 --> 99:59:59.999 Joe 99:59:59.999 --> 99:59:59.999 Director of MIT media lab 99:59:59.999 --> 99:59:59.999 professor of practice. 99:59:59.999 --> 99:59:59.999 Aperson who doesn't really 99:59:59.999 --> 99:59:59.999 need an intro , 99:59:59.999 --> 99:59:59.999 so I'll keep it extremely brief. 99:59:59.999 --> 99:59:59.999 Just by highlighting two 99:59:59.999 --> 99:59:59.999 of the must reads 99:59:59.999 --> 99:59:59.999 from recent months. 99:59:59.999 --> 99:59:59.999 It's an interview he had, 99:59:59.999 --> 99:59:59.999 a conversation actually with 99:59:59.999 --> 99:59:59.999 President Obama 99:59:59.999 --> 99:59:59.999 in the Wired Magazine. 99:59:59.999 --> 99:59:59.999 on the future of the world, 99:59:59.999 --> 99:59:59.999 addressing AI issues 99:59:59.999 --> 99:59:59.999 among other topics. 99:59:59.999 --> 99:59:59.999 And his book, 99:59:59.999 --> 99:59:59.999 Whiplash, which is somehow a 99:59:59.999 --> 99:59:59.999 survival guide for the 99:59:59.999 --> 99:59:59.999 faster future that we're all 99:59:59.999 --> 99:59:59.999 struggling with. I highly 99:59:59.999 --> 99:59:59.999 recommend it as a reading 99:59:59.999 --> 99:59:59.999 I greatly benefited fromi it. 99:59:59.999 --> 99:59:59.999 So, these are not only 2 99:59:59.999 --> 99:59:59.999 amazing thought leaders. 99:59:59.999 --> 99:59:59.999 They're also wonderful collaborators 99:59:59.999 --> 99:59:59.999 and colleagues 99:59:59.999 --> 99:59:59.999 and I have the great privilege 99:59:59.999 --> 99:59:59.999 through 99:59:59.999 --> 99:59:59.999 Berkley and Kline team to work with b 99:59:59.999 --> 99:59:59.999 bothof them as part of 99:59:59.999 --> 99:59:59.999 our recently launched 99:59:59.999 --> 99:59:59.999 joint venture. 99:59:59.999 --> 99:59:59.999 The AI Ethics and Governance 99:59:59.999 --> 99:59:59.999 Initiative. It's just wonderful 99:59:59.999 --> 99:59:59.999 to have you here 99:59:59.999 --> 99:59:59.999 and spend some time 99:59:59.999 --> 99:59:59.999 with all of us 99:59:59.999 --> 99:59:59.999 and share your thoughts 99:59:59.999 --> 99:59:59.999 so thank you very much 99:59:59.999 --> 99:59:59.999 and welcome. 99:59:59.999 --> 99:59:59.999 (applause) 99:59:59.999 --> 99:59:59.999 Thank you, 99:59:59.999 --> 99:59:59.999 first of all 99:59:59.999 --> 99:59:59.999 some of you may be here thinking 99:59:59.999 --> 99:59:59.999 "wait, this isn't the talk 99:59:59.999 --> 99:59:59.999 that I signed up for. 99:59:59.999 --> 99:59:59.999 So to just give you some of 99:59:59.999 --> 99:59:59.999 the prominence of this 99:59:59.999 --> 99:59:59.999 originally I think there 99:59:59.999 --> 99:59:59.999 was a book talk 99:59:59.999 --> 99:59:59.999 that I was going to do 99:59:59.999 --> 99:59:59.999 with Merckman and then 99:59:59.999 --> 99:59:59.999 I said "oh, well, why don't 99:59:59.999 --> 99:59:59.999 we bring somebody else interesting in 99:59:59.999 --> 99:59:59.999 and Josh joined. 99:59:59.999 --> 99:59:59.999 We were going to have a dialoge 99:59:59.999 --> 99:59:59.999 about his book and 99:59:59.999 --> 99:59:59.999 my book. 99:59:59.999 --> 99:59:59.999 And he had a family emeregency 99:59:59.999 --> 99:59:59.999 and coudln't make it. 99:59:59.999 --> 99:59:59.999 I grabbed Jot and also 99:59:59.999 --> 99:59:59.999 realized just as Ers was 99:59:59.999 --> 99:59:59.999 saying, we're doing a lot 99:59:59.999 --> 99:59:59.999 of work with the 99:59:59.999 --> 99:59:59.999 Berkman Center on 99:59:59.999 --> 99:59:59.999 AI and society and I thought 99:59:59.999 --> 99:59:59.999 this would be a 99:59:59.999 --> 99:59:59.999 sufficiently relevant topic to 99:59:59.999 --> 99:59:59.999 what we were going to 99:59:59.999 --> 99:59:59.999 talk about anyway 99:59:59.999 --> 99:59:59.999 so it wouldn't be that 99:59:59.999 --> 99:59:59.999 much false advertising. 99:59:59.999 --> 99:59:59.999 and It was sort of an idea 99:59:59.999 --> 99:59:59.999 that I think relates to my book as well. 99:59:59.999 --> 99:59:59.999 One, I can't remember who it was 99:59:59.999 --> 99:59:59.999 but a well known author told me 99:59:59.999 --> 99:59:59.999 when you give 99:59:59.999 --> 99:59:59.999 book talks, don't explain your 99:59:59.999 --> 99:59:59.999 whole book because then no 99:59:59.999 --> 99:59:59.999 one will have to buy it. 99:59:59.999 --> 99:59:59.999 So this book actually started 99:59:59.999 --> 99:59:59.999 about 4 years ago. 99:59:59.999 --> 99:59:59.999 And we were just wrapping 99:59:59.999 --> 99:59:59.999 it up as we saw a lot of 99:59:59.999 --> 99:59:59.999 this AI, society,controversy/interests 99:59:59.999 --> 99:59:59.999 start. So the book actually 99:59:59.999 --> 99:59:59.999 sort of ends where our 99:59:59.999 --> 99:59:59.999 exploration of AI and society begins. 99:59:59.999 --> 99:59:59.999 So in a way, it overlaps 99:59:59.999 --> 99:59:59.999 what the book is about 99:59:59.999 --> 99:59:59.999 but is sufficently different 99:59:59.999 --> 99:59:59.999 that you have to read the 99:59:59.999 --> 99:59:59.999 book in order to 99:59:59.999 --> 99:59:59.999 understand 99:59:59.999 --> 99:59:59.999 the whole story 99:59:59.999 --> 99:59:59.999 But, let me. 99:59:59.999 --> 99:59:59.999 I'll just start a few remarks 99:59:59.999 --> 99:59:59.999 We'll have .... present some of his work 99:59:59.999 --> 99:59:59.999 and then we'll have a converstaion 99:59:59.999 --> 99:59:59.999 with all of you 99:59:59.999 --> 99:59:59.999 and feel free to interrupt 99:59:59.999 --> 99:59:59.999 and ask questions 99:59:59.999 --> 99:59:59.999 or disagree. 99:59:59.999 --> 99:59:59.999 I think the(stammers) 99:59:59.999 --> 99:59:59.999 I co-taught a class with Jonathan 99:59:59.999 --> 99:59:59.999 in January in the winter semester. 99:59:59.999 --> 99:59:59.999 His tradtional course he teaches 99:59:59.999 --> 99:59:59.999 is called internet and society 99:59:59.999 --> 99:59:59.999 it's a politics and technology 99:59:59.999 --> 99:59:59.999 of control. .... was there 99:59:59.999 --> 99:59:59.999 others were there, it was 99:59:59.999 --> 99:59:59.999 a fun class. 99:59:59.999 --> 99:59:59.999 But one of the sort of 99:59:59.999 --> 99:59:59.999 framing pieces of how we 99:59:59.999 --> 99:59:59.999 talked about this 99:59:59.999 --> 99:59:59.999 was this sort of framing pieces of 99:59:59.999 --> 99:59:59.999 how we talked about this. 99:59:59.999 --> 99:59:59.999 Was this sort of lesigian? 99:59:59.999 --> 99:59:59.999 picture that many of you 99:59:59.999 --> 99:59:59.999 may have seen in 99:59:59.999 --> 99:59:59.999 his book where 99:59:59.999 --> 99:59:59.999 you have law at the top, 99:59:59.999 --> 99:59:59.999 and then you have markets 99:59:59.999 --> 99:59:59.999 on one side. 99:59:59.999 --> 99:59:59.999 and you have norms on the other 99:59:59.999 --> 99:59:59.999 and you have technology 99:59:59.999 --> 99:59:59.999 underneath and you have you in 99:59:59.999 --> 99:59:59.999 the middle 99:59:59.999 --> 99:59:59.999 and some how 99:59:59.999 --> 99:59:59.999 what you are able to do 99:59:59.999 --> 99:59:59.999 is sort of determine by this 99:59:59.999 --> 99:59:59.999 relationship between law. technology - I think technology 99:59:59.999 --> 99:59:59.999 is on top and law is 99:59:59.999 --> 99:59:59.999 down here. 99:59:59.999 --> 99:59:59.999 But anyway, 99:59:59.999 --> 99:59:59.999 somehow these all 99:59:59.999 --> 99:59:59.999 effect eachother. 99:59:59.999 --> 99:59:59.999 so you can create 99:59:59.999 --> 99:59:59.999 technologies that effect 99:59:59.999 --> 99:59:59.999 the law, 99:59:59.999 --> 99:59:59.999 you can create laws that 99:59:59.999 --> 99:59:59.999 effect norms, 99:59:59.999 --> 99:59:59.999 youcan create norms that 99:59:59.999 --> 99:59:59.999 effect technology 99:59:59.999 --> 99:59:59.999 so some 99:59:59.999 --> 99:59:59.999 realtionship between 99:59:59.999 --> 99:59:59.999 norms, markets, law and technology 99:59:59.999 --> 99:59:59.999 is how we need to be thinking 99:59:59.999 --> 99:59:59.999 in order to 99:59:59.999 --> 99:59:59.999 design all of these systems 99:59:59.999 --> 99:59:59.999 so they work well in th 99:59:59.999 --> 99:59:59.999 future. I think one 99:59:59.999 --> 99:59:59.999 of the key reasons why the collaboration 99:59:59.999 --> 99:59:59.999 between MIT and Harper Law School 99:59:59.999 --> 99:59:59.999 Medialab and Berkman 99:59:59.999 --> 99:59:59.999 so important is that 99:59:59.999 --> 99:59:59.999 you kind of have to get all 99:59:59.999 --> 99:59:59.999 of the pieces 99:59:59.999 --> 99:59:59.999 and the people in the same room 99:59:59.999 --> 99:59:59.999 because the problem is 99:59:59.999 --> 99:59:59.999 once everyone has a solution 99:59:59.999 --> 99:59:59.999 and they're trying to convince 99:59:59.999 --> 99:59:59.999 eachother of the solution 99:59:59.999 --> 99:59:59.999 it's, I call them, people 99:59:59.999 --> 99:59:59.999 selling doll houses. 99:59:59.999 --> 99:59:59.999 rather than legos . 99:59:59.999 --> 99:59:59.999 What you want is a whole pile of legos 99:59:59.999 --> 99:59:59.999 with lawyers and business people 99:59:59.999 --> 99:59:59.999 and technologists and policy makers 99:59:59.999 --> 99:59:59.999 playing with the legos 99:59:59.999 --> 99:59:59.999 rather than 99:59:59.999 --> 99:59:59.999 trying to sell eachother thier 99:59:59.999 --> 99:59:59.999 own dollhouses. 99:59:59.999 --> 99:59:59.999 That's what was sort 99:59:59.999 --> 99:59:59.999 of fun with the class 99:59:59.999 --> 99:59:59.999 is that I think 99:59:59.999 --> 99:59:59.999 a lot of lawyers 99:59:59.999 --> 99:59:59.999 realized that actually infact, 99:59:59.999 --> 99:59:59.999 whether you're talking about bit 99:59:59.999 --> 99:59:59.999 coin or differential privacy or AI 99:59:59.999 --> 99:59:59.999 we still have a lot of choices 99:59:59.999 --> 99:59:59.999 to make on the technology side 99:59:59.999 --> 99:59:59.999 and in fact those can 99:59:59.999 --> 99:59:59.999 be informed by 99:59:59.999 --> 99:59:59.999 policy and law 99:59:59.999 --> 99:59:59.999 and conversly, I think 99:59:59.999 --> 99:59:59.999 a lot of the technologists thought 99:59:59.999 --> 99:59:59.999 that law was something like 99:59:59.999 --> 99:59:59.999 laws of physics that just are. But in 99:59:59.999 --> 99:59:59.999 fact laws are the result 99:59:59.999 --> 99:59:59.999 of lawyers and policy 99:59:59.999 --> 99:59:59.999 makers taking to 99:59:59.999 --> 99:59:59.999 technologists. 99:59:59.999 --> 99:59:59.999 Imagining what society wants. 99:59:59.999 --> 99:59:59.999 So we're sort of in the process right 99:59:59.999 --> 99:59:59.999 now of 99:59:59.999 --> 99:59:59.999 struggling through how 99:59:59.999 --> 99:59:59.999 we think about this. 99:59:59.999 --> 99:59:59.999 But Importantly, 99:59:59.999 --> 99:59:59.999 it's already happening 99:59:59.999 --> 99:59:59.999 so it's not like we 99:59:59.999 --> 99:59:59.999 have that much time. 99:59:59.999 --> 99:59:59.999 I think it was Pedro Demingo in 99:59:59.999 --> 99:59:59.999 his book 99:59:59.999 --> 99:59:59.999 says in master algorithm 99:59:59.999 --> 99:59:59.999 and this isn't the exact 99:59:59.999 --> 99:59:59.999 I'm paraphrasing the quote 99:59:59.999 --> 99:59:59.999 it's something like 99:59:59.999 --> 99:59:59.999 I'm less afraid of 99:59:59.999 --> 99:59:59.999 a super intelligence coming 99:59:59.999 --> 99:59:59.999 to take over the world 99:59:59.999 --> 99:59:59.999 and more worried about 99:59:59.999 --> 99:59:59.999 a stupid intelligence that's taken 99:59:59.999 --> 99:59:59.999 over already. 99:59:59.999 --> 99:59:59.999 You know? 99:59:59.999 --> 99:59:59.999 I think that's very close to where we are. 99:59:59.999 --> 99:59:59.999 I think if you see 99:59:59.999 --> 99:59:59.999 Julie A's paper , article in 99:59:59.999 --> 99:59:59.999 Propublic, I guess it was 99:59:59.999 --> 99:59:59.999 a little over a year ago. 99:59:59.999 --> 99:59:59.999 where she happens to find a district where 99:59:59.999 --> 99:59:59.999 they're forced to disclose court records. 99:59:59.999 --> 99:59:59.999 So she was specifically going after the 99:59:59.999 --> 99:59:59.999 fact that machine learning 99:59:59.999 --> 99:59:59.999 AI is now used by the 99:59:59.999 --> 99:59:59.999 judiciary to set bail 99:59:59.999 --> 99:59:59.999 to do parole 99:59:59.999 --> 99:59:59.999 and even sentencing. 99:59:59.999 --> 99:59:59.999 And they have this thing 99:59:59.999 --> 99:59:59.999 called the risk score 99:59:59.999 --> 99:59:59.999 where the machine sort 99:59:59.999 --> 99:59:59.999 of pops up 99:59:59.999 --> 99:59:59.999 after it does an assesment of 99:59:59.999 --> 99:59:59.999 a person's history 99:59:59.999 --> 99:59:59.999 looks at thier interviews and 99:59:59.999 --> 99:59:59.999 she found, and this 99:59:59.999 --> 99:59:59.999 is great, cause she's a 99:59:59.999 --> 99:59:59.999 math matician 99:59:59.999 --> 99:59:59.999 in a data sense. She crunched 99:59:59.999 --> 99:59:59.999 all these numbers 99:59:59.999 --> 99:59:59.999 and it shows 99:59:59.999 --> 99:59:59.999 that for many cases for white people 99:59:59.999 --> 99:59:59.999 is't sort of nearly random 99:59:59.999 --> 99:59:59.999 in some cases. 99:59:59.999 --> 99:59:59.999 So, it's a number but it's still almost random. 99:59:59.999 --> 99:59:59.999 and then for black people 99:59:59.999 --> 99:59:59.999 it's biased against them 99:59:59.999 --> 99:59:59.999 and what's interesting 99:59:59.999 --> 99:59:59.999 is when I talked to ... 99:59:59.999 --> 99:59:59.999 a prosecuter the other day 99:59:59.999 --> 99:59:59.999 he said well I love, they 99:59:59.999 --> 99:59:59.999 love these numbers 99:59:59.999 --> 99:59:59.999 because you get a risk score 99:59:59.999 --> 99:59:59.999 that says okay, 99:59:59.999 --> 99:59:59.999 this person has a 99:59:59.999 --> 99:59:59.999 risk rating of 8 99:59:59.999 --> 99:59:59.999 and so then the court can 99:59:59.999 --> 99:59:59.999 say 'okay, we'll give you this bail 99:59:59.999 --> 99:59:59.999 because the last thing that they 99:59:59.999 --> 99:59:59.999 want is for them to give them 99:59:59.999 --> 99:59:59.999 some bail 99:59:59.999 --> 99:59:59.999 and then the person goes out and murders 99:59:59.999 --> 99:59:59.999 somebody, it's sort of thier fault. 99:59:59.999 --> 99:59:59.999 If they're taken the risk score 99:59:59.999 --> 99:59:59.999 they can say, "I just looked at 99:59:59.999 --> 99:59:59.999 the risk score 99:59:59.999 --> 99:59:59.999 it absolves them of this 99:59:59.999 --> 99:59:59.999 responsibility 99:59:59.999 --> 99:59:59.999 and so there's this 99:59:59.999 --> 99:59:59.999 really interesting question 99:59:59.999 --> 99:59:59.999 that even at random 99:59:59.999 --> 99:59:59.999 it's still, there' s this wierd moral 99:59:59.999 --> 99:59:59.999 hazard that even though you 99:59:59.999 --> 99:59:59.999 have agency, you're able to 99:59:59.999 --> 99:59:59.999 push off 99:59:59.999 --> 99:59:59.999 this responsibility to the machine, right? 99:59:59.999 --> 99:59:59.999 And then you can sort of say, 99:59:59.999 --> 99:59:59.999 well it was math. 99:59:59.999 --> 99:59:59.999 and the problem right now 99:59:59.999 --> 99:59:59.999 is these algorithms are running 99:59:59.999 --> 99:59:59.999 on data sets and rating 99:59:59.999 --> 99:59:59.999 systems that are closed. 99:59:59.999 --> 99:59:59.999 We see this happening in a 99:59:59.999 --> 99:59:59.999 variety of fields . I think we 99:59:59.999 --> 99:59:59.999 see this happening 99:59:59.999 --> 99:59:59.999 in the judiciary, 99:59:59.999 --> 99:59:59.999 Which is a scary place for it 99:59:59.999 --> 99:59:59.999 to be happening 99:59:59.999 --> 99:59:59.999 And so part of this initiative with AI 99:59:59.999 --> 99:59:59.999 fund that we're doing 99:59:59.999 --> 99:59:59.999 we're going to try and look 99:59:59.999 --> 99:59:59.999 at whether we can create 99:59:59.999 --> 99:59:59.999 more transparency and auditability 99:59:59.999 --> 99:59:59.999 we're also seeing it in medicine. 99:59:59.999 --> 99:59:59.999 There's a study that I heard 99:59:59.999 --> 99:59:59.999 that when a doctor overrulled the machine 99:59:59.999 --> 99:59:59.999 in diagnostics the doctor was wrong 99:59:59.999 --> 99:59:59.999 70 % of the time. 99:59:59.999 --> 99:59:59.999 So what does that mean? 99:59:59.999 --> 99:59:59.999 So if you're a doctor 99:59:59.999 --> 99:59:59.999 and you know for a fact 99:59:59.999 --> 99:59:59.999 that you're 70% likely 99:59:59.999 --> 99:59:59.999 on average to be wrong 99:59:59.999 --> 99:59:59.999 are you ever going to overrule 99:59:59.999 --> 99:59:59.999 the machine? 99:59:59.999 --> 99:59:59.999 And what about that 30% 99:59:59.999 --> 99:59:59.999 where the doctors are right? 99:59:59.999 --> 99:59:59.999 So, it creates a very difficult 99:59:59.999 --> 99:59:59.999 situation. 99:59:59.999 --> 99:59:59.999 You look at... 99:59:59.999 --> 99:59:59.999 Imagine war 99:59:59.999 --> 99:59:59.999 We talk about autonomous 99:59:59.999 --> 99:59:59.999 weapons and there's this whole 99:59:59.999 --> 99:59:59.999 fight about 99:59:59.999 --> 99:59:59.999 it, but what if all of the data 99:59:59.999 --> 99:59:59.999 and not what if, 99:59:59.999 --> 99:59:59.999 In fact, all of the data 99:59:59.999 --> 99:59:59.999 that's driving intellegence 99:59:59.999 --> 99:59:59.999 the way that you get on to 99:59:59.999 --> 99:59:59.999 the termination list 99:59:59.999 --> 99:59:59.999 as a target, 99:59:59.999 --> 99:59:59.999 a lot of it involves statistical 99:59:59.999 --> 99:59:59.999 analysis of your activity 99:59:59.999 --> 99:59:59.999 your emotions, your calls 99:59:59.999 --> 99:59:59.999 and there's this great interview 99:59:59.999 --> 99:59:59.999 I think it was in the Independent or 99:59:59.999 --> 99:59:59.999 it was Indpendent. 99:59:59.999 --> 99:59:59.999 There was this guy who 99:59:59.999 --> 99:59:59.999 I think he was in 99:59:59.999 --> 99:59:59.999 Pakistan 99:59:59.999 --> 99:59:59.999 I'm gonna get this wrong 99:59:59.999 --> 99:59:59.999 I'll um, but it's close. 99:59:59.999 --> 99:59:59.999 But he had been attacked 99:59:59.999 --> 99:59:59.999 a number of times 99:59:59.999 --> 99:59:59.999 where the collateral damage 99:59:59.999 --> 99:59:59.999 was family members 99:59:59.999 --> 99:59:59.999 being dead so he knew he 99:59:59.999 --> 99:59:59.999 was on the kill list 99:59:59.999 --> 99:59:59.999 but he didn't know how to get off 99:59:59.999 --> 99:59:59.999 so he goes to London 99:59:59.999 --> 99:59:59.999 to kind of fight for 99:59:59.999 --> 99:59:59.999 'wait, look at me, talk to me 99:59:59.999 --> 99:59:59.999 I'm on this kill list 99:59:59.999 --> 99:59:59.999 but I'm not a bad guy. 99:59:59.999 --> 99:59:59.999 Somehow 99:59:59.999 --> 99:59:59.999 you got the wrong person. 99:59:59.999 --> 99:59:59.999 But there's no interface 99:59:59.999 --> 99:59:59.999 in which he can sort of 99:59:59.999 --> 99:59:59.999 lobby and petition for getting off 99:59:59.999 --> 99:59:59.999 this kill list. 99:59:59.999 --> 99:59:59.999 So even though the person 99:59:59.999 --> 99:59:59.999 controlling the drone strike and 99:59:59.999 --> 99:59:59.999 pushing the button 99:59:59.999 --> 99:59:59.999 maybe a human being, 99:59:59.999 --> 99:59:59.999 If all of the data that's feeding into, or 99:59:59.999 --> 99:59:59.999 a substantial amount of data that's 99:59:59.999 --> 99:59:59.999 feeding into the decision to 99:59:59.999 --> 99:59:59.999 put the person on 99:59:59.999 --> 99:59:59.999 the kill list 99:59:59.999 --> 99:59:59.999 is from a machine, 99:59:59.999 --> 99:59:59.999 I don't know how that's that 99:59:59.999 --> 99:59:59.999 different 99:59:59.999 --> 99:59:59.999 from the machine actually being charged. 99:59:59.999 --> 99:59:59.999 So we talk about sort of these 99:59:59.999 --> 99:59:59.999 future autonomous systems 99:59:59.999 --> 99:59:59.999 and robots running around 99:59:59.999 --> 99:59:59.999 and killing people as 99:59:59.999 --> 99:59:59.999 a sort of scary thing. 99:59:59.999 --> 99:59:59.999 But if we are just pushing a button that the 99:59:59.999 --> 99:59:59.999 robot just tells us to do 99:59:59.999 --> 99:59:59.999 ABC or D but robot 99:59:59.999 --> 99:59:59.999 says it's C, you're going to push C. 99:59:59.999 --> 99:59:59.999 Apparently that was how Kisenger controlled Nixon 99:59:59.999 --> 99:59:59.999 was through his elbow. 99:59:59.999 --> 99:59:59.999 The anwer was always C. 99:59:59.999 --> 99:59:59.999 But anyway, 99:59:59.999 --> 99:59:59.999 the.... that actually is 99:59:59.999 --> 99:59:59.999 when we think about practice. 99:59:59.999 --> 99:59:59.999 We may already be in autonomous 99:59:59.999 --> 99:59:59.999 mode in many things. 99:59:59.999 --> 99:59:59.999 And then I'm going to T up to 99:59:59.999 --> 99:59:59.999 Y? which is one of the 99:59:59.999 --> 99:59:59.999 first places where the rubber meets 99:59:59.999 --> 99:59:59.999 the road is with autonomous vehicles. 99:59:59.999 --> 99:59:59.999 and a lot of the people that 99:59:59.999 --> 99:59:59.999 I talk to say 99:59:59.999 --> 99:59:59.999 that while the real soul searching 99:59:59.999 --> 99:59:59.999 around this is going to happen 99:59:59.999 --> 99:59:59.999 when the next big autonmous vehicle 99:59:59.999 --> 99:59:59.999 accident happens where 99:59:59.999 --> 99:59:59.999 it's clearly the machine's fault, 99:59:59.999 --> 99:59:59.999 How is that going to play out? 99:59:59.999 --> 99:59:59.999 So that may be one of the things 99:59:59.999 --> 99:59:59.999 But the last thing that 99:59:59.999 --> 99:59:59.999 I'll say is that I think 99:59:59.999 --> 99:59:59.999 this is where the media lab 99:59:59.999 --> 99:59:59.999 is excited. I think it's kind of 99:59:59.999 --> 99:59:59.999 an interface design problem 99:59:59.999 --> 99:59:59.999 because part of what the 99:59:59.999 --> 99:59:59.999 problem is is that you may think 99:59:59.999 --> 99:59:59.999 that by pushing , the button, the right to 99:59:59.999 --> 99:59:59.999 overrule the computer the right to 99:59:59.999 --> 99:59:59.999 launch the missle 99:59:59.999 --> 99:59:59.999 may be your finger 99:59:59.999 --> 99:59:59.999 if you have no choice, 99:59:59.999 --> 99:59:59.999 morally or statistically other than 99:59:59.999 --> 99:59:59.999 to push the button 99:59:59.999 --> 99:59:59.999 you're not in charge anymore. Right? 99:59:59.999 --> 99:59:59.999 So what I think we need to think about 99:59:59.999 --> 99:59:59.999 is how to we bring 99:59:59.999 --> 99:59:59.999 society and humans into the decision making 99:59:59.999 --> 99:59:59.999 process so that the answer that 99:59:59.999 --> 99:59:59.999 we derive involves human beings 99:59:59.999 --> 99:59:59.999 and how does that interface hapen? what is the 99:59:59.999 --> 99:59:59.999 right way to do it 99:59:59.999 --> 99:59:59.999 Because I think what we are going to end 99:59:59.999 --> 99:59:59.999 up with is collective decision making. 99:59:59.999 --> 99:59:59.999 machines. and what we want to 99:59:59.999 --> 99:59:59.999 not be in is human agency with no real decision 99:59:59.999 --> 99:59:59.999 making ability. 99:59:59.999 --> 99:59:59.999 And then we can talk more about some 99:59:59.999 --> 99:59:59.999 of the ideas. 99:59:59.999 --> 99:59:59.999 but I'll hand it over to Jot, 99:59:59.999 --> 99:59:59.999 Thank you. 99:59:59.999 --> 99:59:59.999 So I'll just give a short 99:59:59.999 --> 99:59:59.999 overview of 99:59:59.999 --> 99:59:59.999 the research we' ve been doing 99:59:59.999 --> 99:59:59.999 on autonomous vehicles 99:59:59.999 --> 99:59:59.999 I'm not a driverless car expert. 99:59:59.999 --> 99:59:59.999 I don't build driverless cars. 99:59:59.999 --> 99:59:59.999 But I'm interested in them as 99:59:59.999 --> 99:59:59.999 as kind of a social phenomenon 99:59:59.999 --> 99:59:59.999 and the reason has to do with this dilema that 99:59:59.999 --> 99:59:59.999 Steve will keep discussing. 99:59:59.999 --> 99:59:59.999 You know, what if it's an 99:59:59.999 --> 99:59:59.999 autonomous car 99:59:59.999 --> 99:59:59.999 that is going to for some 99:59:59.999 --> 99:59:59.999 reason harm a bunch of 99:59:59.999 --> 99:59:59.999 pedestrians 99:59:59.999 --> 99:59:59.999 crossing the street because the 99:59:59.999 --> 99:59:59.999 brakes are broken 99:59:59.999 --> 99:59:59.999 or because they jumped in front 99:59:59.999 --> 99:59:59.999 of it or whatever. 99:59:59.999 --> 99:59:59.999 But the car can swerve and kill one 99:59:59.999 --> 99:59:59.999 bystander on the other side. 99:59:59.999 --> 99:59:59.999 in order to minimize harm 99:59:59.999 --> 99:59:59.999 in order to save 5 or 10 people 99:59:59.999 --> 99:59:59.999 should the car do this? 99:59:59.999 --> 99:59:59.999 And who should decide? 99:59:59.999 --> 99:59:59.999 And more interestingly, 99:59:59.999 --> 99:59:59.999 what if the car could 99:59:59.999 --> 99:59:59.999 swerve and hit a wall harming 99:59:59.999 --> 99:59:59.999 the passenger or killing the passenger.? 99:59:59.999 --> 99:59:59.999 In order to save these people? 99:59:59.999 --> 99:59:59.999 Should the car do this as well? 99:59:59.999 --> 99:59:59.999 What does the car 99:59:59.999 --> 99:59:59.999 have a duty towards.? 99:59:59.999 --> 99:59:59.999 Minimizing harm?utilitarian principle? 99:59:59.999 --> 99:59:59.999 Protection of the owner or passengers in the car? 99:59:59.999 --> 99:59:59.999 Duty toward them? Or something else? A sort of negotiation 99:59:59.999 --> 99:59:59.999 inbetween? 99:59:59.999 --> 99:59:59.999 Do we ignore this problem? 99:59:59.999 --> 99:59:59.999 do we just say 99:59:59.999 --> 99:59:59.999 well let the car deal with this problem 99:59:59.999 --> 99:59:59.999 and it seems to be a very 99:59:59.999 --> 99:59:59.999 controversial topic because 99:59:59.999 --> 99:59:59.999 there are lots of people 99:59:59.999 --> 99:59:59.999 who love this, 99:59:59.999 --> 99:59:59.999 and lots of people who hate this. 99:59:59.999 --> 99:59:59.999 and people who hate this say, 99:59:59.999 --> 99:59:59.999 Well this is never going to happen 99:59:59.999 --> 99:59:59.999 it's just so statistically unlikely 99:59:59.999 --> 99:59:59.999 and I think that kind 99:59:59.999 --> 99:59:59.999 of misses the point 99:59:59.999 --> 99:59:59.999 because this is a invitro e xploration 99:59:59.999 --> 99:59:59.999 of a principle so you strip 99:59:59.999 --> 99:59:59.999 away all of the things that 99:59:59.999 --> 99:59:59.999 don't matter in the real world 99:59:59.999 --> 99:59:59.999 so you can isolate the factor 99:59:59.999 --> 99:59:59.999 you know, does drug x 99:59:59.999 --> 99:59:59.999 cause this particular 99:59:59.999 --> 99:59:59.999 reation in a cell for example? 99:59:59.999 --> 99:59:59.999 You know, you don't do this in the 99:59:59.999 --> 99:59:59.999 forest you do it in a petridish. 99:59:59.999 --> 99:59:59.999 And this is the petri dish for studying 99:59:59.999 --> 99:59:59.999 human perception of machine ethics 99:59:59.999 --> 99:59:59.999 and what other factors do people 99:59:59.999 --> 99:59:59.999 seem to be ticked off by? 99:59:59.999 --> 99:59:59.999 I think when we started studying 99:59:59.999 --> 99:59:59.999 this we used the 99:59:59.999 --> 99:59:59.999 techniques from social 99:59:59.999 --> 99:59:59.999 psychology, we framed these 99:59:59.999 --> 99:59:59.999 problems to people, 99:59:59.999 --> 99:59:59.999 we varied things, 99:59:59.999 --> 99:59:59.999 the number of people 99:59:59.999 --> 99:59:59.999 who are being sacrified or 99:59:59.999 --> 99:59:59.999 otherwise, whether there's an active 99:59:59.999 --> 99:59:59.999 omission vs act of comission 99:59:59.999 --> 99:59:59.999 and things like this. 99:59:59.999 --> 99:59:59.999 And we're sort of interested in 99:59:59.999 --> 99:59:59.999 how to people want to resolve 99:59:59.999 --> 99:59:59.999 this dilemma? 99:59:59.999 --> 99:59:59.999 What's fascinating is that 99:59:59.999 --> 99:59:59.999 there was somthing that 99:59:59.999 --> 99:59:59.999 was so obvious that we 99:59:59.999 --> 99:59:59.999 missed initially, and that 99:59:59.999 --> 99:59:59.999 was : it's not really 99:59:59.999 --> 99:59:59.999 an ethical question, it's more 99:59:59.999 --> 99:59:59.999 of a social dilemma. So 99:59:59.999 --> 99:59:59.999 it's a question about 99:59:59.999 --> 99:59:59.999 how you negotiate 99:59:59.999 --> 99:59:59.999 the interests of different people. 99:59:59.999 --> 99:59:59.999 And this was the sort of 99:59:59.999 --> 99:59:59.999 strongest finding that we 99:59:59.999 --> 99:59:59.999 found. Which was, no one 99:59:59.999 --> 99:59:59.999 wants to be in a self sacrificing car 99:59:59.999 --> 99:59:59.999 but they want to whole world 99:59:59.999 --> 99:59:59.999 to drive one. 99:59:59.999 --> 99:59:59.999 And it's really fascinating 99:59:59.999 --> 99:59:59.999 that the fact is so strong. 99:59:59.999 --> 99:59:59.999 You know, if you look at 99:59:59.999 --> 99:59:59.999 the morality of sacrifice 99:59:59.999 --> 99:59:59.999 and this is if you kill a pedestrian 99:59:59.999 --> 99:59:59.999 to save ten 99:59:59.999 --> 99:59:59.999 kill a passenger to save ten 99:59:59.999 --> 99:59:59.999 and so forth. 99:59:59.999 --> 99:59:59.999 so you can see that 99:59:59.999 --> 99:59:59.999 I think it's moral and desirable 99:59:59.999 --> 99:59:59.999 in both my car and 99:59:59.999 --> 99:59:59.999 other cars to sacrifice other people 99:59:59.999 --> 99:59:59.999 for the greater good. So I'm 99:59:59.999 --> 99:59:59.999 happy to kill pedestrians to 99:59:59.999 --> 99:59:59.999 save ten that's great 99:59:59.999 --> 99:59:59.999 but as soon as you tell me, 99:59:59.999 --> 99:59:59.999 'well would you sacrifice yourself? 99:59:59.999 --> 99:59:59.999 would you sacrifice your passenger?' 99:59:59.999 --> 99:59:59.999 Well I think it's moral , 99:59:59.999 --> 99:59:59.999 I think it's great, but I would 99:59:59.999 --> 99:59:59.999 never want this in my car. 99:59:59.999 --> 99:59:59.999 Not in other cars and 99:59:59.999 --> 99:59:59.999 defininately not in my car. 99:59:59.999 --> 99:59:59.999 This is where you see these 99:59:59.999 --> 99:59:59.999 things split. 99:59:59.999 --> 99:59:59.999 Now this is the tragedy 99:59:59.999 --> 99:59:59.999 at the commons 99:59:59.999 --> 99:59:59.999 right? 99:59:59.999 --> 99:59:59.999 I want public safety to be maximized. 99:59:59.999 --> 99:59:59.999 I would like the world 99:59:59.999 --> 99:59:59.999 to be a safer place 99:59:59.999 --> 99:59:59.999 where the cars 99:59:59.999 --> 99:59:59.999 might make the decisions that 99:59:59.999 --> 99:59:59.999 minimize harm, but 99:59:59.999 --> 99:59:59.999 I don't want to contribute to 99:59:59.999 --> 99:59:59.999 this public good. I woudln't 99:59:59.999 --> 99:59:59.999 want to pay the personal cost 99:59:59.999 --> 99:59:59.999 needed to do this. 99:59:59.999 --> 99:59:59.999 So we thought 99:59:59.999 --> 99:59:59.999 maybe regulation? You know 99:59:59.999 --> 99:59:59.999 that's how public goods, problems 99:59:59.999 --> 99:59:59.999 are solved. 99:59:59.999 --> 99:59:59.999 Let's set a quota on the 99:59:59.999 --> 99:59:59.999 number of sheep that 99:59:59.999 --> 99:59:59.999 can graze so we 99:59:59.999 --> 99:59:59.999 don't have to overrrun the pasture. 99:59:59.999 --> 99:59:59.999 Or let's set a quota on 99:59:59.999 --> 99:59:59.999 the number of fish you 99:59:59.999 --> 99:59:59.999 can catch so you don't 99:59:59.999 --> 99:59:59.999 over fish and kill 99:59:59.999 --> 99:59:59.999 all the fish and basically 99:59:59.999 --> 99:59:59.999 everybody loses out. 99:59:59.999 --> 99:59:59.999 And we ask people whether 99:59:59.999 --> 99:59:59.999 they would support this 99:59:59.999 --> 99:59:59.999 and we found that people 99:59:59.999 --> 99:59:59.999 think it's moral , but 99:59:59.999 --> 99:59:59.999 they don't want it to be 99:59:59.999 --> 99:59:59.999 legally enforced. 99:59:59.999 --> 99:59:59.999 At least for now. Right? 99:59:59.999 --> 99:59:59.999 This is the PR 99:59:59.999 --> 99:59:59.999 problem and maybe it's a.. 99:59:59.999 --> 99:59:59.999 maybe we need to double up 99:59:59.999 --> 99:59:59.999 the law 99:59:59.999 --> 99:59:59.999 so that people can 99:59:59.999 --> 99:59:59.999 feel comfortable with 99:59:59.999 --> 99:59:59.999 what this means. 99:59:59.999 --> 99:59:59.999 (audience)-I'd just like to ask a question ? 99:59:59.999 --> 99:59:59.999 because we talk a lot about the 99:59:59.999 --> 99:59:59.999 evolution of cultural things 99:59:59.999 --> 99:59:59.999 and I assume all of these are people, 99:59:59.999 --> 99:59:59.999 or I guess you don't know 99:59:59.999 --> 99:59:59.999 but most of these are 99:59:59.999 --> 99:59:59.999 people who have never 99:59:59.999 --> 99:59:59.999 been in a self driving car 99:59:59.999 --> 99:59:59.999 right? And I think one of the 99:59:59.999 --> 99:59:59.999 things we found , again, this is not 99:59:59.999 --> 99:59:59.999 my work but some of our colleagues 99:59:59.999 --> 99:59:59.999 They do this self driving car, 99:59:59.999 --> 99:59:59.999 Uber type thing where you can 99:59:59.999 --> 99:59:59.999 map and it was actually for 99:59:59.999 --> 99:59:59.999 normal sort of, the public. 99:59:59.999 --> 99:59:59.999 And they're impression 99:59:59.999 --> 99:59:59.999 of the safety of self drving cars 99:59:59.999 --> 99:59:59.999 changed substantially 99:59:59.999 --> 99:59:59.999 after they had expereinced 99:59:59.999 --> 99:59:59.999 it for a little while 99:59:59.999 --> 99:59:59.999 and they sort of anticdotally 99:59:59.999 --> 99:59:59.999 felt safer than with dad. 99:59:59.999 --> 99:59:59.999 So I think once you're in 99:59:59.999 --> 99:59:59.999 a self driving car, and 99:59:59.999 --> 99:59:59.999 see how much control it 99:59:59.999 --> 99:59:59.999 99:59:59.999 --> 99:59:59.999 has your view on the safety 99:59:59.999 --> 99:59:59.999 as well as it's, 99:59:59.999 --> 99:59:59.999 and the other thing that happens and 99:59:59.999 --> 99:59:59.999 this may happen more 99:59:59.999 --> 99:59:59.999 in Japan than the US 99:59:59.999 --> 99:59:59.999 In Japenese culture 99:59:59.999 --> 99:59:59.999 your sort of identify with 99:59:59.999 --> 99:59:59.999 machines and tools like that 99:59:59.999 --> 99:59:59.999 they start to feel trust with 99:59:59.999 --> 99:59:59.999 the machine 99:59:59.999 --> 99:59:59.999 which I think unless you expereince it 99:59:59.999 --> 99:59:59.999 you don't, you can't imagine it. 99:59:59.999 --> 99:59:59.999 Anyway. 99:59:59.999 --> 99:59:59.999 I agree. 99:59:59.999 --> 99:59:59.999 I think there's all sorts of 99:59:59.999 --> 99:59:59.999 things . We're now interested in 99:59:59.999 --> 99:59:59.999 studying, for example, 99:59:59.999 --> 99:59:59.999 agency perception. You know, 99:59:59.999 --> 99:59:59.999 do people see these things to 99:59:59.999 --> 99:59:59.999 have minds, and if not, why not? 99:59:59.999 --> 99:59:59.999 what's the missing component? 99:59:59.999 --> 99:59:59.999 Which becomes really interesting 99:59:59.999 --> 99:59:59.999 with drones for example. 99:59:59.999 --> 99:59:59.999 So the other thing is 99:59:59.999 --> 99:59:59.999 when we ask people, well again, 99:59:59.999 --> 99:59:59.999 people think it's moral 99:59:59.999 --> 99:59:59.999 to sacrifice but they don't want it 99:59:59.999 --> 99:59:59.999 to be regulated 99:59:59.999 --> 99:59:59.999 and definately not 99:59:59.999 --> 99:59:59.999 buy it if it's regulated 99:59:59.999 --> 99:59:59.999 but they're much more likely to 99:59:59.999 --> 99:59:59.999 purchase those cars if they were regulated. 99:59:59.999 --> 99:59:59.999 I think this is really a really important 99:59:59.999 --> 99:59:59.999 question. 99:59:59.999 --> 99:59:59.999 If people don't purchase those cars 99:59:59.999 --> 99:59:59.999 you will not save lives. 99:59:59.999 --> 99:59:59.999 I mean people estimate, 99:59:59.999 --> 99:59:59.999 scientists estimate that 90 % of 99:59:59.999 --> 99:59:59.999 accidents today are due to human 99:59:59.999 --> 99:59:59.999 error. So if we can , the sooner, assuming 99:59:59.999 --> 99:59:59.999 the technology get's there assuming 99:59:59.999 --> 99:59:59.999 we have wide adoption 99:59:59.999 --> 99:59:59.999 the sooner we save more lives. 99:59:59.999 --> 99:59:59.999 but if the people 99:59:59.999 --> 99:59:59.999 are so worried about edge cases 99:59:59.999 --> 99:59:59.999 or that their own safety is not paramount 99:59:59.999 --> 99:59:59.999 they may not purchase the 99:59:59.999 --> 99:59:59.999 cars and we may not therefor 99:59:59.999 --> 99:59:59.999 have wide adoption and as a result 99:59:59.999 --> 99:59:59.999 - we can map this onto 99:59:59.999 --> 99:59:59.999 the quadrants this is clearly one 99:59:59.999 --> 99:59:59.999 that you can't just leave up to the market. 99:59:59.999 --> 99:59:59.999 If people aren't buying 99:59:59.999 --> 99:59:59.999 the thing that they believe 99:59:59.999 --> 99:59:59.999 has a common good. 99:59:59.999 --> 99:59:59.999 -Exactly, and if you regulate it 99:59:59.999 --> 99:59:59.999 you can, there's a backfire 99:59:59.999 --> 99:59:59.999 effect which is well fine, 99:59:59.999 --> 99:59:59.999 that's great, that's a good 99:59:59.999 --> 99:59:59.999 social contract for other 99:59:59.999 --> 99:59:59.999 people but I will continue to 99:59:59.999 --> 99:59:59.999 drive my own car 99:59:59.999 --> 99:59:59.999 and probably be more 99:59:59.999 --> 99:59:59.999 likely to kill myself 99:59:59.999 --> 99:59:59.999 as a result. 99:59:59.999 --> 99:59:59.999 So people are not rational 99:59:59.999 --> 99:59:59.999 in they way they 99:59:59.999 --> 99:59:59.999 assess risk of getting on a plane 99:59:59.999 --> 99:59:59.999 or will I be eaten by a shark? You know 99:59:59.999 --> 99:59:59.999 people over estimate 99:59:59.999 --> 99:59:59.999 those risks and there's a 99:59:59.999 --> 99:59:59.999 good chance that 99:59:59.999 --> 99:59:59.999 if we don't trust those 99:59:59.999 --> 99:59:59.999 systems then we will overestimate 99:59:59.999 --> 99:59:59.999 those risks too and 99:59:59.999 --> 99:59:59.999 prefer to drive ourselves. 99:59:59.999 --> 99:59:59.999 so we have an ethical 99:59:59.999 --> 99:59:59.999 dilemma we strarted from 99:59:59.999 --> 99:59:59.999 then we realized it's a 99:59:59.999 --> 99:59:59.999 social dilemma 99:59:59.999 --> 99:59:59.999 but now we're realizing 99:59:59.999 --> 99:59:59.999 there's a meta ethical dilemma 99:59:59.999 --> 99:59:59.999 which is if you solve the 99:59:59.999 --> 99:59:59.999 social dilemma 99:59:59.999 --> 99:59:59.999 by using regualtions 99:59:59.999 --> 99:59:59.999 you may actually create a 99:59:59.999 --> 99:59:59.999 bigger dilemma 99:59:59.999 --> 99:59:59.999 a bigger trolley problem 99:59:59.999 --> 99:59:59.999 which is do we continue 99:59:59.999 --> 99:59:59.999 to drive cars or sell or 99:59:59.999 --> 99:59:59.999 do we lead to wide adoption? 99:59:59.999 --> 99:59:59.999 of autonomous vehicles. 99:59:59.999 --> 99:59:59.999 so we want to collect more data. 99:59:59.999 --> 99:59:59.999 we want to understand this 99:59:59.999 --> 99:59:59.999 issue in more nuanced ways 99:59:59.999 --> 99:59:59.999 and we started, I'm gonna 99:59:59.999 --> 99:59:59.999 move fast on this 99:59:59.999 --> 99:59:59.999 we started collecting data, these 99:59:59.999 --> 99:59:59.999 things have made it 99:59:59.999 --> 99:59:59.999 to transoportation regulations now 99:59:59.999 --> 99:59:59.999 or guidelines which is good 99:59:59.999 --> 99:59:59.999 but we've created a website 99:59:59.999 --> 99:59:59.999 called moral machine 99:59:59.999 --> 99:59:59.999 in which we randomly 99:59:59.999 --> 99:59:59.999 generate scenarios 99:59:59.999 --> 99:59:59.999 so in this case it's not just 99:59:59.999 --> 99:59:59.999 one vs ten 99:59:59.999 --> 99:59:59.999 or one vs five 99:59:59.999 --> 99:59:59.999 it's there's a dog in there 99:59:59.999 --> 99:59:59.999 and we've varied the ages, 99:59:59.999 --> 99:59:59.999 sometimes they're children 99:59:59.999 --> 99:59:59.999 sometimes they're pregnant 99:59:59.999 --> 99:59:59.999 woman 99:59:59.999 --> 99:59:59.999 sometimes people are crossing at red 99:59:59.999 --> 99:59:59.999 lights 99:59:59.999 --> 99:59:59.999 and so do they deserve the same 99:59:59.999 --> 99:59:59.999 level of protection as -isn't 99:59:59.999 --> 99:59:59.999 that interesting? 99:59:59.999 --> 99:59:59.999 This group here, 99:59:59.999 --> 99:59:59.999 what if they're children? 99:59:59.999 --> 99:59:59.999 Do they, should they, 99:59:59.999 --> 99:59:59.999 are they expected to know 99:59:59.999 --> 99:59:59.999 not to cross the red light? 99:59:59.999 --> 99:59:59.999 and so it get's really hairy 99:59:59.999 --> 99:59:59.999 really quickly. 99:59:59.999 --> 99:59:59.999 You know these are 99:59:59.999 --> 99:59:59.999 some cartoons 99:59:59.999 --> 99:59:59.999 very simplified scenarios 99:59:59.999 --> 99:59:59.999 but i think they still 99:59:59.999 --> 99:59:59.999 bring out lots of interesting 99:59:59.999 --> 99:59:59.999 questions and we show 99:59:59.999 --> 99:59:59.999 people the resulfs 99:59:59.999 --> 99:59:59.999 (audience groaning at result) 99:59:59.999 --> 99:59:59.999 We show people results. 99:59:59.999 --> 99:59:59.999 This is a former of mine 99:59:59.999 --> 99:59:59.999 who has a cat 99:59:59.999 --> 99:59:59.999 he's happy to kill babies 99:59:59.999 --> 99:59:59.999 to save cats 99:59:59.999 --> 99:59:59.999 we also show people, 99:59:59.999 --> 99:59:59.999 we show people how much they 99:59:59.999 --> 99:59:59.999 care about different factors 99:59:59.999 --> 99:59:59.999 and how that compared to with others. 99:59:59.999 --> 99:59:59.999 So people love this 99:59:59.999 --> 99:59:59.999 cause it's kind of metal to thier 99:59:59.999 --> 99:59:59.999 own morality. 99:59:59.999 --> 99:59:59.999 You know, do I care about 99:59:59.999 --> 99:59:59.999 the law a lot and how 99:59:59.999 --> 99:59:59.999 do I 99:59:59.999 --> 99:59:59.999 compare with other people on this 99:59:59.999 --> 99:59:59.999 matter? Do I protect passengers more 99:59:59.999 --> 99:59:59.999 than other people or less,? 99:59:59.999 --> 99:59:59.999 and so on. 99:59:59.999 --> 99:59:59.999 We also have this design mode 99:59:59.999 --> 99:59:59.999 where people can create thier own 99:59:59.999 --> 99:59:59.999 scenarios and they get a link 99:59:59.999 --> 99:59:59.999 to them and 99:59:59.999 --> 99:59:59.999 and a lot of people have been using 99:59:59.999 --> 99:59:59.999 them to teach ethics 99:59:59.999 --> 99:59:59.999 in highschools 99:59:59.999 --> 99:59:59.999 and universities. 99:59:59.999 --> 99:59:59.999 And we have all sorts of you know, 99:59:59.999 --> 99:59:59.999 species preferences 99:59:59.999 --> 99:59:59.999 should social value be taken 99:59:59.999 --> 99:59:59.999 into account 99:59:59.999 --> 99:59:59.999 should age be taken into 99:59:59.999 --> 99:59:59.999 account and so forth. 99:59:59.999 --> 99:59:59.999 And we also evaluate if there 99:59:59.999 --> 99:59:59.999 is an omission or comission 99:59:59.999 --> 99:59:59.999 distinction. 99:59:59.999 --> 99:59:59.999 which action, is the action that minimizes 99:59:59.999 --> 99:59:59.999 harm should it be an omission or commision 99:59:59.999 --> 99:59:59.999 and there is definitely a bias. 99:59:59.999 --> 99:59:59.999 WE're now analyses. 99:59:59.999 --> 99:59:59.999 So far we've translated this into 99:59:59.999 --> 99:59:59.999 10 languages we've recieved 99:59:59.999 --> 99:59:59.999 3 million uses 99:59:59.999 --> 99:59:59.999 that have completed more than 99:59:59.999 --> 99:59:59.999 28 million decisions 99:59:59.999 --> 99:59:59.999 binary choices a 99:59:59.999 --> 99:59:59.999 and we have 300000 fulll 99:59:59.999 --> 99:59:59.999 surveys 99:59:59.999 --> 99:59:59.999 and this is still growing fast 99:59:59.999 --> 99:59:59.999 These full serveys 99:59:59.999 --> 99:59:59.999 allow us to tease out 99:59:59.999 --> 99:59:59.999 whether these people 99:59:59.999 --> 99:59:59.999 have cars 99:59:59.999 --> 99:59:59.999 themselves, which age bracket, 99:59:59.999 --> 99:59:59.999 which income bracket 99:59:59.999 --> 99:59:59.999 they come from 99:59:59.999 --> 99:59:59.999 and so on. 99:59:59.999 --> 99:59:59.999 This is really interesting because then you can 99:59:59.999 --> 99:59:59.999 start 99:59:59.999 --> 99:59:59.999 saying well people how 99:59:59.999 --> 99:59:59.999 have cars maybe 99:59:59.999 --> 99:59:59.999 more or less likely 99:59:59.999 --> 99:59:59.999 to support this particular 99:59:59.999 --> 99:59:59.999 ethical framework. 99:59:59.999 --> 99:59:59.999 We have a lot of global coverage and so far we've 99:59:59.999 --> 99:59:59.999 been looking it 99:59:59.999 --> 99:59:59.999 cross cultural differences 99:59:59.999 --> 99:59:59.999 and because this is recorded 99:59:59.999 --> 99:59:59.999 I don't want to talk about it 99:59:59.999 --> 99:59:59.999 yet but basically 99:59:59.999 --> 99:59:59.999 we're observing some very 99:59:59.999 --> 99:59:59.999 interesting cross cultural differences 99:59:59.999 --> 99:59:59.999 in terms of the degree to 99:59:59.999 --> 99:59:59.999 which people are utilitarian 99:59:59.999 --> 99:59:59.999 or to which thy would prioritize 99:59:59.999 --> 99:59:59.999 the passengers 99:59:59.999 --> 99:59:59.999 to which they're willing to take 99:59:59.999 --> 99:59:59.999 an action so omission vs 99:59:59.999 --> 99:59:59.999 comission and so forth 99:59:59.999 --> 99:59:59.999 I think it's really facinating 99:59:59.999 --> 99:59:59.999 and it would be very important 99:59:59.999 --> 99:59:59.999 precondition to any sort of 99:59:59.999 --> 99:59:59.999 public relations effort to make the 99:59:59.999 --> 99:59:59.999 cars more acceptable 99:59:59.999 --> 99:59:59.999 but also potentially to the legal differenes in the 99:59:59.999 --> 99:59:59.999 legal frameworks 99:59:59.999 --> 99:59:59.999 as well. 99:59:59.999 --> 99:59:59.999 Also beginning to look at partial autonomy, 99:59:59.999 --> 99:59:59.999 so whether it's autonomous cars or 99:59:59.999 --> 99:59:59.999 drones 99:59:59.999 --> 99:59:59.999 or jujges making bail decisions 99:59:59.999 --> 99:59:59.999 again, you can have a 99:59:59.999 --> 99:59:59.999 machine to everything or 99:59:59.999 --> 99:59:59.999 you can have a human do everything. 99:59:59.999 --> 99:59:59.999 so and in the car you 99:59:59.999 --> 99:59:59.999 have things where the driver 99:59:59.999 --> 99:59:59.999 assitance so the person is 99:59:59.999 --> 99:59:59.999 in control and the machine 99:59:59.999 --> 99:59:59.999 sort of watches over them 99:59:59.999 --> 99:59:59.999 so tpyota has been promoting 99:59:59.999 --> 99:59:59.999 this model and other car makers as well 99:59:59.999 --> 99:59:59.999 but also there 99:59:59.999 --> 99:59:59.999 s auto pilot where the machine does 99:59:59.999 --> 99:59:59.999 the things and thehuman 99:59:59.999 --> 99:59:59.999 kind of has to keep an eye on 99:59:59.999 --> 99:59:59.999 whether it's a car or anything else and 99:59:59.999 --> 99:59:59.999 then you have full autonomy. 99:59:59.999 --> 99:59:59.999 The question here we're interested in 99:59:59.999 --> 99:59:59.999 is we're comparing these 99:59:59.999 --> 99:59:59.999 models and we're investigating 99:59:59.999 --> 99:59:59.999 empirically whether 99:59:59.999 --> 99:59:59.999 people assign 99:59:59.999 --> 99:59:59.999 differnt degrees of blame 99:59:59.999 --> 99:59:59.999 and responsibility depending on the 99:59:59.999 --> 99:59:59.999 control architecture, we can call it 99:59:59.999 --> 99:59:59.999 If a person overrriding a decision made by 99:59:59.999 --> 99:59:59.999 a machine is differnt from a machine 99:59:59.999 --> 99:59:59.999 overriding a decision made by a human 99:59:59.999 --> 99:59:59.999 and it happens 99:59:59.999 --> 99:59:59.999 again this is now in submission, 99:59:59.999 --> 99:59:59.999 But it happens to really matter. 99:59:59.999 --> 99:59:59.999 It really matters who you think is 99:59:59.999 --> 99:59:59.999 ultimately responsible and who is liable 99:59:59.999 --> 99:59:59.999 and I think this is a sort of psychological 99:59:59.999 --> 99:59:59.999 imput to potential legistations that 99:59:59.999 --> 99:59:59.999 could come up. 99:59:59.999 --> 99:59:59.999 to deal with these scenarios 99:59:59.999 --> 99:59:59.999 so this is a broader picture 99:59:59.999 --> 99:59:59.999 that I like. wich I think Joey 99:59:59.999 --> 99:59:59.999 eluded to initially. 99:59:59.999 --> 99:59:59.999 which is that there is a gap in 99:59:59.999 --> 99:59:59.999 between 99:59:59.999 --> 99:59:59.999 and on one side we have engineers 99:59:59.999 --> 99:59:59.999 who think everything is an engeneering 99:59:59.999 --> 99:59:59.999 problem, you know, ev 99:59:59.999 --> 99:59:59.999 everythin can be enginerred away 99:59:59.999 --> 99:59:59.999 and you have people from 99:59:59.999 --> 99:59:59.999 the humanities and social sciences 99:59:59.999 --> 99:59:59.999 study the nuances of human 99:59:59.999 --> 99:59:59.999 behavior 99:59:59.999 --> 99:59:59.999 but also who know how rules 99:59:59.999 --> 99:59:59.999 can get sort of abused 99:59:59.999 --> 99:59:59.999 and have a good sort of knack 99:59:59.999 --> 99:59:59.999 for this. You know, how do you 99:59:59.999 --> 99:59:59.999 ensure you have coherent system of 99:59:59.999 --> 99:59:59.999 ethics and values and checks and 99:59:59.999 --> 99:59:59.999 balances and so on 99:59:59.999 --> 99:59:59.999 I think that these sides often don't 99:59:59.999 --> 99:59:59.999 talk to each other so I think 99:59:59.999 --> 99:59:59.999 there is a sizable community of 99:59:59.999 --> 99:59:59.999 people who complain about who 99:59:59.999 --> 99:59:59.999 are very good identifying a problem 99:59:59.999 --> 99:59:59.999 and violations of fairness and rights. 99:59:59.999 --> 99:59:59.999 and so on 99:59:59.999 --> 99:59:59.999 but we don't have the tools 99:59:59.999 --> 99:59:59.999 to express these objections 99:59:59.999 --> 99:59:59.999 in a way that computer scientists 99:59:59.999 --> 99:59:59.999 can operationalize 99:59:59.999 --> 99:59:59.999 likewise, we have machine 99:59:59.999 --> 99:59:59.999 learning 99:59:59.999 --> 99:59:59.999 and there are scientist who feel 99:59:59.999 --> 99:59:59.999 that this is problamatic 99:59:59.999 --> 99:59:59.999 I can see that this has, can cause 99:59:59.999 --> 99:59:59.999 problems this can 99:59:59.999 --> 99:59:59.999 violate 99:59:59.999 --> 99:59:59.999 people's rights but again 99:59:59.999 --> 99:59:59.999 they don't have the intellectual 99:59:59.999 --> 99:59:59.999 framework to raise these issues in 99:59:59.999 --> 99:59:59.999 a way that humans as a society 99:59:59.999 --> 99:59:59.999 can evaluate so we're hoping 99:59:59.999 --> 99:59:59.999 to do and this is part of the partnership 99:59:59.999 --> 99:59:59.999 between the media lab and the berkman center 99:59:59.999 --> 99:59:59.999 Berkman center are from this side and they 99:59:59.999 --> 99:59:59.999 understand us, 99:59:59.999 --> 99:59:59.999 and we come from this side from technology and we 99:59:59.999 --> 99:59:59.999 work on interfaces` 99:59:59.999 --> 99:59:59.999 and we hope through this 99:59:59.999 --> 99:59:59.999 we will make some interesting famework 99:59:59.999 --> 99:59:59.999 this is , I think, where 99:59:59.999 --> 99:59:59.999 many of the interesting questions are. 99:59:59.999 --> 99:59:59.999 So I think we're ready for a discussion and taking some 99:59:59.999 --> 99:59:59.999 questions. 99:59:59.999 --> 99:59:59.999 "I guess the one other part I would add 99:59:59.999 --> 99:59:59.999 to this is that 99:59:59.999 --> 99:59:59.999 just one other acess is going back 99:59:59.999 --> 99:59:59.999 to judiciary but we can have 99:59:59.999 --> 99:59:59.999 this 99:59:59.999 --> 99:59:59.999 in cars as well. Is in the one hand 99:59:59.999 --> 99:59:59.999 I don't think anybody 99:59:59.999 --> 99:59:59.999 thinks that speeding tickets issues 99:59:59.999 --> 99:59:59.999 by speed cameras on the highway 99:59:59.999 --> 99:59:59.999 are , I mean some people may 99:59:59.999 --> 99:59:59.999 not like them but 99:59:59.999 --> 99:59:59.999 that's inappropriate use of machine 99:59:59.999 --> 99:59:59.999 because it's really a fact. There's a speed 99:59:59.999 --> 99:59:59.999 that you're allowed to go. and the 99:59:59.999 --> 99:59:59.999 machine is more likely to measure your s 99:59:59.999 --> 99:59:59.999 speed than 99:59:59.999 --> 99:59:59.999 a human eye balling it, 99:59:59.999 --> 99:59:59.999 and probably more fair. 99:59:59.999 --> 99:59:59.999 on the other hand I don't think anyone believes 99:59:59.999 --> 99:59:59.999 that the supreme court 99:59:59.999 --> 99:59:59.999 decisions at least for now should 99:59:59.999 --> 99:59:59.999 have really that much substantial role 99:59:59.999 --> 99:59:59.999 with the machines at least in the deliberation 99:59:59.999 --> 99:59:59.999 part so there's a spectrum there's this 99:59:59.999 --> 99:59:59.999 thing where on the one end 99:59:59.999 --> 99:59:59.999 where you're just establishing a fact which 99:59:59.999 --> 99:59:59.999 is sort of impemation of the law 99:59:59.999 --> 99:59:59.999 which we're not even disputing 99:59:59.999 --> 99:59:59.999 the justice of it to 99:59:59.999 --> 99:59:59.999 the supreme court which 99:59:59.999 --> 99:59:59.999 is supposed to try and reflect the norms 99:59:59.999 --> 99:59:59.999 of the day making 99:59:59.999 --> 99:59:59.999 determinations about laws. 99:59:59.999 --> 99:59:59.999 but then there's a continum between 99:59:59.999 --> 99:59:59.999 somewhere in the middle there 99:59:59.999 --> 99:59:59.999 you have this uncanny place where 99:59:59.999 --> 99:59:59.999 it feels like the machines have some 99:59:59.999 --> 99:59:59.999 infulence and I think whats 99:59:59.999 --> 99:59:59.999 kind of interesting is just about all 99:59:59.999 --> 99:59:59.999 of these hypotheticals we have 99:59:59.999 --> 99:59:59.999 there's one extreme where you do want 99:59:59.999 --> 99:59:59.999 the machines in charge, 99:59:59.999 --> 99:59:59.999 there's another extreme where you 99:59:59.999 --> 99:59:59.999 do want humans in charge. 99:59:59.999 --> 99:59:59.999 Those are actually not that difficult. 99:59:59.999 --> 99:59:59.999 Ther's a space inbetween them 99:59:59.999 --> 99:59:59.999 and I thikn that's why it's kind of 99:59:59.999 --> 99:59:59.999 an interface problem 99:59:59.999 --> 99:59:59.999 is that it's very unclear how the human 99:59:59.999 --> 99:59:59.999 and machine peices whether it's a societal thing 99:59:59.999 --> 99:59:59.999 or an individual get together so that's 99:59:59.999 --> 99:59:59.999 again, it's related to the autonomy question 99:59:59.999 --> 99:59:59.999 but I think it's a .. and I think it's 99:59:59.999 --> 99:59:59.999 sort of technology and is it ethics or 99:59:59.999 --> 99:59:59.999 morality there's some sort of stack as well. 99:59:59.999 --> 99:59:59.999 And maybe everything to an internet person 99:59:59.999 --> 99:59:59.999 looks like a stack. 99:59:59.999 --> 99:59:59.999 Maybe that's my proble. 99:59:59.999 --> 99:59:59.999 I there's sort of an intereting thought 99:59:59.999 --> 99:59:59.999 experiment which was 99:59:59.999 --> 99:59:59.999 you know, suppose that we 99:59:59.999 --> 99:59:59.999 I think we need tools too, it's not 99:59:59.999 --> 99:59:59.999 just a legal question. New kinds of tools 99:59:59.999 --> 99:59:59.999 and new kinds of data can make a 99:59:59.999 --> 99:59:59.999 big difference. So let's assume 99:59:59.999 --> 99:59:59.999 that we invented the cars 99:59:59.999 --> 99:59:59.999 and they started going on high speed 99:59:59.999 --> 99:59:59.999 but we didn't invent radar. 99:59:59.999 --> 99:59:59.999 that can accurately measure speeds. 99:59:59.999 --> 99:59:59.999 so we relied on human guestimation of speeds 99:59:59.999 --> 99:59:59.999 of your speed driving. So there's 99:59:59.999 --> 99:59:59.999 a police man standing sort of 99:59:59.999 --> 99:59:59.999 eyeballing cars and (thinks) 99:59:59.999 --> 99:59:59.999 that sort of looks like 120 right? 99:59:59.999 --> 99:59:59.999 You can very well imagine that 99:59:59.999 --> 99:59:59.999 under this scenario , if policemen 99:59:59.999 --> 99:59:59.999 were discriminating one particular group 99:59:59.999 --> 99:59:59.999 maybe over estimate the speed of 99:59:59.999 --> 99:59:59.999 people driving cars from 99:59:59.999 --> 99:59:59.999 that particular ethinic group. 99:59:59.999 --> 99:59:59.999 and underestimate the speed of 99:59:59.999 --> 99:59:59.999 other people. 99:59:59.999 --> 99:59:59.999 But somehow the tool solves this 99:59:59.999 --> 99:59:59.999 question because it makes the 99:59:59.999 --> 99:59:59.999 final youknow, it's recorded 99:59:59.999 --> 99:59:59.999 and somehow it becomes objective. 99:59:59.999 --> 99:59:59.999 It becomes a fact. 99:59:59.999 --> 99:59:59.999 And we haven't, you know, 99:59:59.999 --> 99:59:59.999 It's not disputeable so can we do 99:59:59.999 --> 99:59:59.999 something 99:59:59.999 --> 99:59:59.999 similar here and we say 99:59:59.999 --> 99:59:59.999 -But I think that's where it hits a slippery slope. 99:59:59.999 --> 99:59:59.999 So if you're doing speed of a car 99:59:59.999 --> 99:59:59.999 it's a very small number of data points the 99:59:59.999 --> 99:59:59.999 machine is getting ot guess 99:59:59.999 --> 99:59:59.999 your speed but the risk rating 99:59:59.999 --> 99:59:59.999 to some people may seem very 99:59:59.999 --> 99:59:59.999 scientific especially if they don't 99:59:59.999 --> 99:59:59.999 understand math and statistics 99:59:59.999 --> 99:59:59.999 and so they may say the machine 99:59:59.999 --> 99:59:59.999 rating said they have a risk of this 99:59:59.999 --> 99:59:59.999 and actually in the forms, they never 99:59:59.999 --> 99:59:59.999 ask you your race. 99:59:59.999 --> 99:59:59.999 It just turns out that when you 99:59:59.999 --> 99:59:59.999 collect the data and you collect 99:59:59.999 --> 99:59:59.999 the questions the result is biased 99:59:59.999 --> 99:59:59.999 against race and so 99:59:59.999 --> 99:59:59.999 one of the questions of what's difficult 99:59:59.999 --> 99:59:59.999 is if you don'ot understand how these 99:59:59.999 --> 99:59:59.999 algorithms convert data inot result 99:59:59.999 --> 99:59:59.999 and this is the problem with the 99:59:59.999 --> 99:59:59.999 black box thing. A lot of 99:59:59.999 --> 99:59:59.999 the machines and again there's progress 99:59:59.999 --> 99:59:59.999 on making machines that can explain how 99:59:59.999 --> 99:59:59.999 they got to the decision 99:59:59.999 --> 99:59:59.999 but a lot of the machines we 99:59:59.999 --> 99:59:59.999 currently use are unable to 99:59:59.999 --> 99:59:59.999 describe how they got the number, 99:59:59.999 --> 99:59:59.999 they just give you the number. 99:59:59.999 --> 99:59:59.999 So if I may pick up on that and 99:59:59.999 --> 99:59:59.999 ask a first question? 99:59:59.999 --> 99:59:59.999 So., this question of the normatitvity 99:59:59.999 --> 99:59:59.999 of the autonomous system and who makes 99:59:59.999 --> 99:59:59.999 where's the sorce of the norm that 99:59:59.999 --> 99:59:59.999 seems to be a key question. And I'm wondering 99:59:59.999 --> 99:59:59.999 picking up on your earlier description whether 99:59:59.999 --> 99:59:59.999 we're on a particular trajectory 99:59:59.999 --> 99:59:59.999 by what you described. I think 99:59:59.999 --> 99:59:59.999 there'are roughly 3 phases I've heard. 99:59:59.999 --> 99:59:59.999 1 Is okay, we have these autonomous vehicles 99:59:59.999 --> 99:59:59.999 and now it's a question for law 99:59:59.999 --> 99:59:59.999 makers and regulators 99:59:59.999 --> 99:59:59.999 Do we apply existing norms to 99:59:59.999 --> 99:59:59.999 these new technologies ?. 99:59:59.999 --> 99:59:59.999 Sometimes you need to update 99:59:59.999 --> 99:59:59.999 the regualations which we 99:59:59.999 --> 99:59:59.999 see happening 99:59:59.999 --> 99:59:59.999 you made reference to that. But 99:59:59.999 --> 99:59:59.999 there is also a second phase 99:59:59.999 --> 99:59:59.999 it seems where it, is can we 99:59:59.999 --> 99:59:59.999 somehow program some of the values and laws and 99:59:59.999 --> 99:59:59.999 rules into the systems themselves so the behavior 99:59:59.999 --> 99:59:59.999 is closer to what we have normative constance around 99:59:59.999 --> 99:59:59.999 the society and as law makers and policy 99:59:59.999 --> 99:59:59.999 makers? 99:59:59.999 --> 99:59:59.999 And then there' s a potential third phase 99:59:59.999 --> 99:59:59.999 I'm particulary interested in your views 99:59:59.999 --> 99:59:59.999 whether that is indeed a trajectory 99:59:59.999 --> 99:59:59.999 in the area you study or 99:59:59.999 --> 99:59:59.999 more broadly potentially 99:59:59.999 --> 99:59:59.999 that as you could envision a future 99:59:59.999 --> 99:59:59.999 where more and more data 99:59:59.999 --> 99:59:59.999 accumulates in systems like 99:59:59.999 --> 99:59:59.999 autonomous vehicles based on 99:59:59.999 --> 99:59:59.999 the rules we program then 99:59:59.999 --> 99:59:59.999 how to behave and how they learn 99:59:59.999 --> 99:59:59.999 how these rules are obeyed or not 99:59:59.999 --> 99:59:59.999 what the compliance rate is and the like 99:59:59.999 --> 99:59:59.999 where suddenly the norm itsself becomes 99:59:59.999 --> 99:59:59.999 computer or machine generated and how to do 99:59:59.999 --> 99:59:59.999 we feel about that because that 99:59:59.999 --> 99:59:59.999 may be an advertantly get us to the other end of the 99:59:59.999 --> 99:59:59.999 spectrum that you're decribing that the 99:59:59.999 --> 99:59:59.999 norms are no longer developed here 99:59:59.999 --> 99:59:59.999 and then somehow programmed into the 99:59:59.999 --> 99:59:59.999 system 99:59:59.999 --> 99:59:59.999 but at least evolution of the norm 99:59:59.999 --> 99:59:59.999 happens in the automated system. 99:59:59.999 --> 99:59:59.999 I think you would have to tease apart the 99:59:59.999 --> 99:59:59.999 norms and the laws 99:59:59.999 --> 99:59:59.999 One of my -says the engeneer to the law student - 99:59:59.999 --> 99:59:59.999 (laughing) 99:59:59.999 --> 99:59:59.999 My favorite one is a chart that Jot gave me 99:59:59.999 --> 99:59:59.999 but I did this in japan so I can 99:59:59.999 --> 99:59:59.999 do this here. 99:59:59.999 --> 99:59:59.999 Imagine you have a car 99:59:59.999 --> 99:59:59.999 and on oyur left there's two motor 99:59:59.999 --> 99:59:59.999 cycles one the left and on the right 99:59:59.999 --> 99:59:59.999 the one on the left has no 99:59:59.999 --> 99:59:59.999 helmet, the one on the right 99:59:59.999 --> 99:59:59.999 is wearing a helmet. And there's 99:59:59.999 --> 99:59:59.999 a helmet law. The guy on the left is clearly 99:59:59.999 --> 99:59:59.999 breaking the law completely disrespecting the law 99:59:59.999 --> 99:59:59.999 so you have to sweve. Someone jumps in 99:59:59.999 --> 99:59:59.999 front of your car. You have to swerve. 99:59:59.999 --> 99:59:59.999 Do you hit the guy iwthout the helmet 99:59:59.999 --> 99:59:59.999 or do you hit the guy with the helmet?? 99:59:59.999 --> 99:59:59.999 The guy with the helmet is more likely to survive, 99:59:59.999 --> 99:59:59.999 but he's following the law. 99:59:59.999 --> 99:59:59.999 So who hits the guy without the helmet. 99:59:59.999 --> 99:59:59.999 I did this at a Japenese car company 99:59:59.999 --> 99:59:59.999 and half of them in the room 99:59:59.999 --> 99:59:59.999 raised their hand said well, 99:59:59.999 --> 99:59:59.999 of course you go after the guy who 99:59:59.999 --> 99:59:59.999 broke the law right? 99:59:59.999 --> 99:59:59.999 But this is a very interesting normative 99:59:59.999 --> 99:59:59.999 question 99:59:59.999 --> 99:59:59.999 so there's all these versions of it