0:00:00.843,0:00:03.434 In 2007, I became the attorney general 0:00:03.434,0:00:05.159 of the state of New Jersey. 0:00:05.159,0:00:07.439 Before that, I'd been a criminal prosecutor, 0:00:07.439,0:00:10.120 first in the Manhattan district attorney's office, 0:00:10.120,0:00:12.770 and then at the United States Department of Justice. 0:00:12.770,0:00:14.971 But when I became the attorney general, 0:00:14.971,0:00:18.866 two things happened that changed [br]the way I see criminal justice. 0:00:18.866,0:00:20.896 The first is that I asked what I thought 0:00:20.896,0:00:23.082 were really basic questions. 0:00:23.082,0:00:25.938 I wanted to understand who we were arresting, 0:00:25.938,0:00:27.602 who we were charging, 0:00:27.602,0:00:29.730 and who we were putting in our nation's jails 0:00:29.730,0:00:31.146 and prisons. 0:00:31.146,0:00:32.794 I also wanted to understand 0:00:32.794,0:00:34.123 if we were making decisions 0:00:34.123,0:00:36.641 in a way that made us safer. 0:00:36.641,0:00:39.893 And I couldn't get this information out. 0:00:39.893,0:00:43.250 It turned out that most big criminal justice agencies 0:00:43.250,0:00:44.552 like my own 0:00:44.552,0:00:46.934 didn't track the things that matter. 0:00:46.934,0:00:50.252 So after about a month of being incredibly frustrated, 0:00:50.252,0:00:52.223 I walked down into a conference room 0:00:52.223,0:00:54.113 that was filled with detectives 0:00:54.113,0:00:56.895 and stacks and stacks of case files, 0:00:56.895,0:00:58.071 and the detectives were sitting there 0:00:58.071,0:01:00.305 with yellow legal pads taking notes. 0:01:00.305,0:01:01.891 They were trying to get the information 0:01:01.891,0:01:03.109 I was looking for 0:01:03.109,0:01:05.154 by going through case by case 0:01:05.154,0:01:07.052 for the past five years. 0:01:07.052,0:01:08.705 And as you can imagine, 0:01:08.705,0:01:11.348 when we finally got the results, they weren't good. 0:01:11.348,0:01:13.003 It turned out that we were doing 0:01:13.003,0:01:15.023 a lot of low-level drug cases 0:01:15.023,0:01:16.498 on the streets just around the corner 0:01:16.498,0:01:18.766 from our office in Trenton. 0:01:18.766,0:01:20.233 The second thing that happened 0:01:20.233,0:01:23.907 is that I spent the day in the Camden,[br]New Jersey police department. 0:01:23.907,0:01:25.794 Now, at that time, Camden, New Jersey, 0:01:25.794,0:01:28.446 was the most dangerous city in America. 0:01:28.446,0:01:32.273 I ran the Camden Police[br]Department because of that. 0:01:32.273,0:01:34.385 I spent the day in the police department, 0:01:34.385,0:01:37.111 and I was taken into a room[br]with senior police officials, 0:01:37.111,0:01:38.786 all of whom were working hard 0:01:38.786,0:01:42.043 and trying very hard to reduce crime in Camden. 0:01:42.043,0:01:43.869 And what I saw in that room, 0:01:43.869,0:01:46.114 as we talked about how to reduce crime, 0:01:46.114,0:01:49.973 were a series of officers with a[br]lot of little yellow sticky notes. 0:01:49.973,0:01:52.819 And they would take a yellow sticky[br]and they would write something on it 0:01:52.823,0:01:54.622 and they would put it up on a board. 0:01:54.622,0:01:56.793 And one of them said, [br]"We had a robbery two weeks ago. 0:01:56.793,0:01:58.504 We have no suspects." 0:01:58.504,0:02:03.531 And another said, "We had a shooting in this neighborhood last week. We have no suspects." 0:02:03.531,0:02:06.114 We weren't using data-driven policing. 0:02:06.114,0:02:08.156 We were essentially trying to fight crime 0:02:08.156,0:02:10.683 with yellow Post-it notes. 0:02:10.683,0:02:12.818 Now, both of these things made me realize 0:02:12.818,0:02:16.069 fundamentally that we were failing. 0:02:16.069,0:02:19.192 We didn't even know who was[br]in our criminal justice system, 0:02:19.192,0:02:22.427 we didn't have any data about[br]the things that mattered, 0:02:22.427,0:02:24.995 and we didn't share data or use analytics 0:02:24.995,0:02:27.146 or tools to help us make better decisions 0:02:27.146,0:02:29.149 and to reduce crime. 0:02:29.149,0:02:31.373 And for the first time, I started to think 0:02:31.373,0:02:33.283 about how we made decisions. 0:02:33.283,0:02:34.680 When I was an assistant D.A., 0:02:34.680,0:02:36.550 and when I was a federal prosecutor, 0:02:36.550,0:02:38.296 I looked at the cases in front of me, 0:02:38.296,0:02:40.922 and I generally made decisions based on my instinct 0:02:40.922,0:02:42.614 and my experience. 0:02:42.614,0:02:44.273 When I became attorney general, 0:02:44.273,0:02:45.912 I could look at the system as a whole, 0:02:45.912,0:02:47.730 and what surprised me is that I found 0:02:47.730,0:02:49.635 that that was exactly how we were doing it 0:02:49.635,0:02:51.938 across the entire system -- 0:02:51.938,0:02:54.339 in police departments, in prosecutors's offices, 0:02:54.339,0:02:57.139 in courts and in jails. 0:02:57.139,0:02:59.336 And what I learned very quickly 0:02:59.336,0:03:02.969 is that we weren't doing a good job. 0:03:02.969,0:03:04.985 So I wanted to do things differently. 0:03:04.985,0:03:07.182 I wanted to introduce data and analytics 0:03:07.182,0:03:09.231 and rigorous statistical analysis 0:03:09.231,0:03:10.631 into our work. 0:03:10.631,0:03:13.601 In short, I wanted to moneyball criminal justice. 0:03:13.601,0:03:15.628 Now, moneyball, as many of you know, 0:03:15.628,0:03:17.197 is what the Oakland A's did, 0:03:17.197,0:03:19.170 where they used smart data and statistics 0:03:19.170,0:03:20.792 to figure out how to pick players 0:03:20.792,0:03:22.313 that would help them win games, 0:03:22.313,0:03:25.293 and they went from a system that [br]was based on baseball scouts 0:03:25.293,0:03:27.153 who used to go out and watch players 0:03:27.153,0:03:28.790 and use their instinct and experience, 0:03:28.790,0:03:30.533 the scouts' instincts and experience, 0:03:30.533,0:03:32.246 to pick players, from one to use 0:03:32.246,0:03:35.068 smart data and rigorous statistical analysis 0:03:35.068,0:03:38.439 to figure out how to pick players[br]that would help them win games. 0:03:38.439,0:03:40.237 It worked for the Oakland A's, 0:03:40.237,0:03:42.456 and it worked in the state of New Jersey. 0:03:42.456,0:03:44.529 We took Camden off the top of the list 0:03:44.529,0:03:46.700 as the most dangerous city in America. 0:03:46.700,0:03:49.855 We reduced murders there by 41 percent, 0:03:49.855,0:03:52.837 which actually means 37 lives were saved. 0:03:52.837,0:03:56.577 And we reduced all crime in the city by 26 percent. 0:03:56.577,0:03:59.816 We also changed the way[br]we did criminal prosecutions. 0:03:59.816,0:04:01.821 So we went from doing low-level drug crimes 0:04:01.821,0:04:03.463 that were outside our building 0:04:03.463,0:04:05.805 to doing cases of statewide importance, 0:04:05.805,0:04:08.963 on things like reducing violence[br]with the most violent offenders, 0:04:08.963,0:04:10.821 prosecuting street gangs, 0:04:10.821,0:04:14.229 gun and drug trafficking, and political corruption. 0:04:14.229,0:04:16.731 And all of this matters greatly, 0:04:16.731,0:04:18.676 because public safety to me 0:04:18.676,0:04:21.212 is the most important function of government. 0:04:21.212,0:04:23.510 If we're not safe, we can't be educated, 0:04:23.510,0:04:24.858 we can't be healthy, 0:04:24.858,0:04:27.803 we can't do any of the other things[br]we want to do in our lives. 0:04:27.803,0:04:29.504 And we live in a country today 0:04:29.504,0:04:32.638 where we face serious criminal justice problems. 0:04:32.638,0:04:36.299 We have 12 million arrests every single year. 0:04:36.299,0:04:38.342 The vast majority of those arrests 0:04:38.342,0:04:41.354 are for low-level crimes, like misdemeanors, 0:04:41.354,0:04:43.088 70 to 80 percent. 0:04:43.088,0:04:45.079 Less than five percent of all arrests 0:04:45.079,0:04:46.974 are for violent crime. 0:04:46.974,0:04:49.029 Yet we spend 75 billion, 0:04:49.029,0:04:50.447 that's b for billion, 0:04:50.447,0:04:54.574 dollars a year on state and local corrections costs. 0:04:54.574,0:04:57.415 Right now, today, we have 2.3 million people 0:04:57.415,0:04:59.315 in our jails and prisons. 0:04:59.315,0:05:02.111 And we face unbelievable public safety challenges 0:05:02.111,0:05:04.050 because we have a situation 0:05:04.050,0:05:06.948 in which two thirds of the people in our jails 0:05:06.948,0:05:08.702 are there waiting for trial. 0:05:08.702,0:05:10.837 They haven't yet been convicted of a crime. 0:05:10.837,0:05:12.956 They're just waiting for their day in court. 0:05:12.956,0:05:16.504 And 67 percent of people come back. 0:05:16.504,0:05:19.532 Our recidivism rate is amongst [br]the highest in the world. 0:05:19.532,0:05:21.635 Almost seven in 10 people who are released 0:05:21.635,0:05:23.286 from prison will be rearrested 0:05:23.286,0:05:27.241 in a constant cycle of crime and incarceration. 0:05:27.241,0:05:29.823 So when I started my job at the Arnold Foundation, 0:05:29.823,0:05:32.559 I came back to looking at a lot of these questions, 0:05:32.559,0:05:34.213 and I came back to thinking about how 0:05:34.213,0:05:36.596 we had used data and analytics to transform 0:05:36.596,0:05:39.180 the way we did criminal justice in New Jersey. 0:05:39.180,0:05:41.324 And when I look at the criminal justice system 0:05:41.324,0:05:42.980 in the United States today, 0:05:42.980,0:05:44.619 I feel the exact same way that I did 0:05:44.619,0:05:47.085 about the state of New Jersey when I started there, 0:05:47.085,0:05:50.313 which is that we absolutely have to do better, 0:05:50.313,0:05:52.236 and I know that we can do better. 0:05:52.236,0:05:53.941 So I decided to focus 0:05:53.941,0:05:56.158 on using data and analytics 0:05:56.158,0:05:58.519 to help make the most critical decision 0:05:58.519,0:06:00.125 in public safety, 0:06:00.125,0:06:02.146 and that decision is the determination 0:06:02.146,0:06:04.681 of whether, when someone has been arrested, 0:06:04.681,0:06:06.596 whether they pose a risk to public safety 0:06:06.596,0:06:08.122 and should be detained, 0:06:08.122,0:06:10.478 or whether they don't pose a risk to public safety 0:06:10.478,0:06:12.115 and should be released. 0:06:12.115,0:06:14.034 Everything that happens in criminal cases 0:06:14.034,0:06:15.806 comes out of this one decision. 0:06:15.806,0:06:17.302 It impacts everything. 0:06:17.302,0:06:18.652 It impacts sentencing. 0:06:18.652,0:06:20.553 It impacts whether someone gets drug treatment. 0:06:20.553,0:06:22.876 It impacts crime and violence. 0:06:22.876,0:06:24.813 And when I talk to judges around the United States, 0:06:24.813,0:06:26.741 which I do all the time now, 0:06:26.741,0:06:28.578 they all say the same thing, 0:06:28.578,0:06:31.685 which is that we put dangerous people in jail, 0:06:31.685,0:06:35.210 and we let non-dangerous, nonviolent people out. 0:06:35.210,0:06:37.443 They mean it and they believe it. 0:06:37.443,0:06:39.176 But when you start to look at the data, 0:06:39.176,0:06:41.640 which, by the way, the judges don't have, 0:06:41.640,0:06:43.252 when we start to look at the data, 0:06:43.252,0:06:45.670 what we find time and time again, 0:06:45.670,0:06:47.652 is that this isn't the case. 0:06:47.652,0:06:49.333 We find low-risk offenders, 0:06:49.333,0:06:53.047 which makes up 50 percent of our[br]entire criminal justice population, 0:06:53.047,0:06:55.446 we find that they're in jail. 0:06:55.446,0:06:57.932 Take Leslie Chew, who was a Texas man 0:06:57.932,0:07:00.816 who stole four blankets on a cold winter night. 0:07:00.816,0:07:03.411 He was arrested, and he was kept in jail 0:07:03.411,0:07:05.464 on 3,500 dollars bail, 0:07:05.464,0:07:08.240 an amount that he could not afford to pay. 0:07:08.240,0:07:10.828 And he stayed in jail for eight months 0:07:10.828,0:07:12.893 until his case came up for trial, 0:07:12.893,0:07:16.798 at a cost to taxpayers of more than 9,000 dollars. 0:07:16.798,0:07:18.795 And at the other end of the spectrum, 0:07:18.795,0:07:21.077 we're doing an equally terrible job. 0:07:21.077,0:07:22.649 The people who we find 0:07:22.649,0:07:24.668 are the highest-risk offenders, 0:07:24.668,0:07:27.165 the people who we think have the highest likelihood 0:07:27.165,0:07:29.117 of committing a new crime if they're released, 0:07:29.117,0:07:32.067 we see nationally that 50 percent of those people 0:07:32.067,0:07:34.041 are being released. 0:07:34.041,0:07:37.215 The reason for this is the way we make decisions. 0:07:37.215,0:07:38.924 Judges have the best intentions 0:07:38.924,0:07:40.876 when they make these decisions about risk, 0:07:40.876,0:07:43.360 but they're making them subjectively. 0:07:43.360,0:07:45.506 They're like the baseball scouts 20 years ago 0:07:45.506,0:07:47.637 who were using their instinct and their experience 0:07:47.637,0:07:50.316 to try to decide what risk someone poses. 0:07:50.316,0:07:51.846 They're being subjective, 0:07:51.846,0:07:54.906 and we know what happens[br]with subjective decision making, 0:07:54.906,0:07:57.649 which is that we are often wrong. 0:07:57.649,0:07:59.032 What we need in this space 0:07:59.032,0:08:01.584 are strong data and analytics. 0:08:01.584,0:08:03.331 What I decided to look for 0:08:03.331,0:08:06.167 was a strong data and analytic risk assessment tool, 0:08:06.167,0:08:08.931 something that would let judges actually understand 0:08:08.931,0:08:11.190 with a scientific and objective way 0:08:11.190,0:08:12.837 what the risk was that was posed 0:08:12.837,0:08:14.447 by someone in front of them. 0:08:14.447,0:08:16.096 I looked all over the country, 0:08:16.096,0:08:18.038 and I found that between five and 10 percent 0:08:18.038,0:08:19.367 of all U.S. jurisdictions 0:08:19.367,0:08:22.345 actually use any type of risk assessment tool, 0:08:22.345,0:08:23.970 and when I looked at these tools, 0:08:23.970,0:08:25.830 I quickly realized why. 0:08:25.830,0:08:28.520 They were unbelievably expensive to administer, 0:08:28.520,0:08:30.048 they were time-consuming, 0:08:30.048,0:08:32.155 they were limited to the local jurisdiction 0:08:32.155,0:08:33.585 in which they'd been created. 0:08:33.585,0:08:35.378 So basically, they couldn't be scaled 0:08:35.378,0:08:37.587 or transferred to other places. 0:08:37.587,0:08:39.824 So I went out and built a phenomenal team 0:08:39.824,0:08:41.868 of data scientists and researchers 0:08:41.868,0:08:43.494 and statisticians 0:08:43.494,0:08:46.339 to build a universal risk assessment tool, 0:08:46.339,0:08:48.732 so that every single judge in[br]the United States of America 0:08:48.732,0:08:53.056 can have an objective, scientific measure of risk. 0:08:53.056,0:08:54.714 In the tool that we've built, 0:08:54.714,0:08:57.582 what we did was we collected 1.5 million cases 0:08:57.582,0:08:59.280 from all around the United States, 0:08:59.280,0:09:00.924 from cities, from counties, 0:09:00.924,0:09:02.435 from every single state in the country, 0:09:02.435,0:09:04.181 the federal districts. 0:09:04.181,0:09:06.370 And with those 1.5 million cases, 0:09:06.370,0:09:08.310 which is the largest data set on pretrial 0:09:08.310,0:09:10.115 in the United States today, 0:09:10.115,0:09:11.980 we were able to basically find that there were 0:09:11.980,0:09:15.302 900-plus risk factors that we could look at 0:09:15.302,0:09:18.168 to try to figure out what mattered most. 0:09:18.168,0:09:20.249 And we found that there were nine specific things 0:09:20.249,0:09:22.484 that mattered all across the country 0:09:22.484,0:09:25.461 and that were the most highly predictive of risk. 0:09:25.461,0:09:29.166 And so we built a universal risk assessment tool. 0:09:29.166,0:09:30.611 And it looks like this. 0:09:30.611,0:09:33.223 As you'll see, we put some information in, 0:09:33.223,0:09:35.236 but most of it is incredibly simple, 0:09:35.236,0:09:36.668 it's easy to use, 0:09:36.668,0:09:39.637 it focuses on things like the[br]defendant's prior convictions, 0:09:39.637,0:09:41.616 whether they've been sentenced to incarceration, 0:09:41.616,0:09:43.880 whether they've engaged in violence before, 0:09:43.880,0:09:46.273 whether they've even failed to come back to court. 0:09:46.273,0:09:48.773 And with this tool, we can predict three things. 0:09:48.773,0:09:50.626 First, whether or not someone will commit 0:09:50.626,0:09:52.191 a new crime if they're released. 0:09:52.191,0:09:53.855 Second, for the first time, 0:09:53.855,0:09:55.716 and I think this is incredibly important, 0:09:55.716,0:09:57.639 we can predict whether someone will commit 0:09:57.639,0:09:59.473 an act of violence if they're released. 0:09:59.473,0:10:01.360 And that's the single most important thing 0:10:01.360,0:10:03.167 that judges say when you talk to them. 0:10:03.167,0:10:04.995 And third, we can predict whether someone 0:10:04.995,0:10:06.985 will come back to court. 0:10:06.985,0:10:10.018 And every single judge in the[br]United States of America can use it, 0:10:10.018,0:10:13.830 because it's been created on a universal data set. 0:10:13.830,0:10:16.439 What judges see if they run the risk assessment tool 0:10:16.439,0:10:18.559 is this -- it's a dashboard. 0:10:18.559,0:10:21.407 At the top, you see the New Criminal Activity Score, 0:10:21.407,0:10:23.336 six of course being the highest, 0:10:23.336,0:10:25.739 and then in the middle you[br]see, "Elevated risk of violence." 0:10:25.739,0:10:27.485 What that says is that this person 0:10:27.485,0:10:29.545 is someone who has an elevated risk of violence 0:10:29.545,0:10:31.430 that the judge should look twice at. 0:10:31.430,0:10:32.766 And then, towards the bottom, 0:10:32.766,0:10:34.734 you see the Failure to Appear Score, 0:10:34.734,0:10:36.126 which again is the likelihood 0:10:36.126,0:10:39.139 that someone will come back to court. 0:10:39.139,0:10:41.352 Now I want to say something really important. 0:10:41.352,0:10:44.079 It's not that I think we should be eliminating 0:10:44.079,0:10:46.323 the judge's instinct and experience 0:10:46.323,0:10:47.927 from this process. 0:10:47.927,0:10:48.985 I don't. 0:10:48.985,0:10:50.992 I actually believe the problem that we see 0:10:50.992,0:10:53.846 and the reason that we have[br]these incredible system errors, 0:10:53.846,0:10:56.933 where we're incarcerating[br]low-level, nonviolent people 0:10:56.933,0:11:00.105 and we're releasing high-risk, dangerous people, 0:11:00.105,0:11:02.828 is that we don't have an objective measure of risk. 0:11:02.828,0:11:04.128 But what I believe should happen 0:11:04.128,0:11:06.928 is that we should take that[br]data-driven risk assessment 0:11:06.928,0:11:09.969 and combine that with the[br]judge's instinct and experience 0:11:09.969,0:11:12.927 to lead us to better decision making. 0:11:12.927,0:11:16.230 The tool went statewide in Kentucky on July 1, 0:11:16.230,0:11:19.581 and we're about to go up in a[br]number of other U.S. jurisdictions. 0:11:19.581,0:11:22.172 Our goal, quite simply, is that every single judge 0:11:22.172,0:11:24.364 in the United States will use a data-driven risk tool 0:11:24.364,0:11:26.455 within the next five years. 0:11:26.455,0:11:27.807 We're now working on risk tools 0:11:27.807,0:11:31.091 for prosecutors and for police officers as well, 0:11:31.091,0:11:33.791 to try to take a system that runs today 0:11:33.791,0:11:36.587 in America the same way it did 50 years ago, 0:11:36.587,0:11:38.684 based on instinct and experience, 0:11:38.684,0:11:40.539 and make it into one that runs 0:11:40.539,0:11:43.008 on data and analytics. 0:11:43.008,0:11:44.929 Now, the great news about all this, 0:11:44.929,0:11:46.546 and we have a ton of work left to do, 0:11:46.546,0:11:48.403 and we have a lot of culture to change, 0:11:48.403,0:11:50.149 but the great news about all of it 0:11:50.149,0:11:52.017 is that we know it works. 0:11:52.017,0:11:54.170 It's why Google is Google, 0:11:54.170,0:11:56.632 and it's why all these baseball teams use moneyball 0:11:56.632,0:11:58.413 to win games. 0:11:58.413,0:12:00.150 The great news for us as well 0:12:00.150,0:12:02.046 is that it's the way that we can transform 0:12:02.046,0:12:04.367 the American criminal justice system. 0:12:04.367,0:12:06.724 It's how we can make our streets safer, 0:12:06.724,0:12:09.023 we can reduce our prison costs, 0:12:09.023,0:12:11.090 and we can make our system much fairer 0:12:11.090,0:12:12.815 and more just. 0:12:12.815,0:12:14.977 Some people call it data science. 0:12:14.977,0:12:17.278 I call it moneyballing criminal justice. 0:12:17.278,0:12:19.082 Thank you. 0:12:19.082,0:12:23.175 (Applause)