0:00:00.843,0:00:03.158
In 2007, I became the Attorney General

0:00:03.158,0:00:05.159
of the State of New Jersey.

0:00:05.159,0:00:07.439
Before that, I'd been a criminal prosecutor,

0:00:07.439,0:00:10.120
first in the Manhattan District Attorney's office,

0:00:10.120,0:00:12.770
and then at the United States Department of Justice.

0:00:12.770,0:00:14.980
But when I became the Attorney General,

0:00:14.980,0:00:15.925
two things happened

0:00:15.925,0:00:18.866
that changed the way I see criminal justice.

0:00:18.866,0:00:20.589
The first is that I asked what I thought

0:00:20.589,0:00:23.082
were really basic questions.

0:00:23.082,0:00:26.246
I wanted to understand who we were arresting,

0:00:26.246,0:00:27.433
who we were charging,

0:00:27.433,0:00:29.730
and who we were putting in our nation's jails

0:00:29.730,0:00:31.146
and prisons.

0:00:31.146,0:00:32.794
I also wanted to understand

0:00:32.794,0:00:34.123
if we were making decisions

0:00:34.123,0:00:36.641
in a way that made us safer.

0:00:36.641,0:00:39.893
And I couldn't get this information out.

0:00:39.893,0:00:43.250
It turned out that most big criminal justice agencies

0:00:43.250,0:00:44.552
like my own

0:00:44.552,0:00:46.934
didn't track the things that matter.

0:00:46.934,0:00:50.252
So after about a month of being incredibly frustrated,

0:00:50.252,0:00:52.223
I walked down into a conference room

0:00:52.223,0:00:54.113
that was filled with detectives

0:00:54.113,0:00:56.895
and stacks and stacks of case files,

0:00:56.895,0:00:58.071
and the detectives were sitting there

0:00:58.071,0:01:00.305
with yellow legal pads taking notes.

0:01:00.305,0:01:01.845
They were trying to get the information

0:01:01.845,0:01:03.109
I was looking for

0:01:03.109,0:01:05.154
by going through case by case

0:01:05.154,0:01:07.591
for the past five years.

0:01:07.591,0:01:08.705
And as you can imagine,

0:01:08.705,0:01:11.795
when we finally got the results, they weren't good.

0:01:11.795,0:01:13.003
It turned out that we were doing

0:01:13.003,0:01:15.023
a lot of low-level drug cases

0:01:15.023,0:01:16.498
on the streets just around the corner

0:01:16.498,0:01:18.766
from our office in Trenton.

0:01:18.766,0:01:20.233
The second thing that happened

0:01:20.233,0:01:23.907
is that I spent the day in the Camden,[br]New Jersey, Police Department.

0:01:23.907,0:01:25.794
Now at that time, Camden, New Jersey,

0:01:25.794,0:01:28.446
was the most dangerous city in America.

0:01:28.446,0:01:32.273
I ran the Camden Police[br]Department because of that.

0:01:32.273,0:01:34.385
I spent the day in the Police Department,

0:01:34.385,0:01:37.111
and I was taken into a room[br]with senior police officials,

0:01:37.111,0:01:38.786
all of whom were working hard

0:01:38.786,0:01:42.320
and trying very hard to reduce crime in Camden.

0:01:42.320,0:01:43.869
And what I saw in that room,

0:01:43.869,0:01:46.114
as we talked about how to reduce crime,

0:01:46.114,0:01:49.973
were a series of officers with a[br]lot of little yellow sticky notes.

0:01:49.973,0:01:51.804
And they would take a yellow sticky

0:01:51.804,0:01:52.823
and they would write something on it

0:01:52.823,0:01:54.622
and they would put it up on a board.

0:01:54.622,0:01:56.717
And one of them said, [br]"We had a robbery two weeks ago.

0:01:56.717,0:01:58.504
We have no suspects."

0:01:58.504,0:02:03.531
And another said, "We had a shooting in this neighborhood last week. We have no suspects."

0:02:03.531,0:02:06.114
We weren't using data-driven policing.

0:02:06.114,0:02:08.156
We were essentially trying to fight crime

0:02:08.156,0:02:10.683
with yellow post-it notes.

0:02:10.683,0:02:12.818
Now both of these things made me realize

0:02:12.818,0:02:16.069
fundamentally that we were failing.

0:02:16.069,0:02:19.192
We didn't even know who was[br]in our criminal justice system,

0:02:19.192,0:02:22.427
we didn't have any data about[br]the things that mattered,

0:02:22.427,0:02:24.995
and we didn't share data or use analytics

0:02:24.995,0:02:27.146
or tools to help us make better decisions

0:02:27.146,0:02:29.149
and to reduce crime.

0:02:29.149,0:02:31.127
And for the first time, I started to think

0:02:31.127,0:02:33.283
about how we made decisions.

0:02:33.283,0:02:34.680
When I was an assistant D.A.,

0:02:34.680,0:02:36.550
and when I was a federal prosecutor,

0:02:36.550,0:02:38.296
I looked at the cases in front of me,

0:02:38.296,0:02:40.922
and I generally made decisions based on my instinct

0:02:40.922,0:02:42.614
and my experience.

0:02:42.614,0:02:44.273
When I became Attorney General,

0:02:44.273,0:02:45.912
I could look at this system as a whole,

0:02:45.912,0:02:47.377
and what surprised me is that I found

0:02:47.377,0:02:49.635
that that was exactly how we were doing it

0:02:49.635,0:02:51.938
across the entire system,

0:02:51.938,0:02:54.339
in police departments, in prosecutors's offices,

0:02:54.339,0:02:57.139
in courts, and in jails.

0:02:57.139,0:02:59.336
And what I learned very quickly

0:02:59.336,0:03:02.969
is that we weren't doing a good job.

0:03:02.969,0:03:04.985
So I wanted to do things differently.

0:03:04.985,0:03:07.182
I wanted to introduce data and analytics

0:03:07.182,0:03:09.231
and rigorous statistical analysis

0:03:09.231,0:03:10.631
into our work.

0:03:10.631,0:03:13.601
In short, I wanted to moneyball criminal justice.

0:03:13.601,0:03:15.628
Now, moneyball, as many of you know,

0:03:15.628,0:03:17.197
is what the Oakland A's did,

0:03:17.197,0:03:19.170
where they used smart data and statistics

0:03:19.170,0:03:20.469
to figure out how to pick players

0:03:20.469,0:03:22.313
that would help them win games,

0:03:22.313,0:03:24.298
and they went from a system that was based

0:03:24.298,0:03:25.386
on baseball scouts

0:03:25.386,0:03:27.153
who used to go out and watch players

0:03:27.153,0:03:28.714
and use their instinct and experience,

0:03:28.714,0:03:30.533
the scouts' instincts and experience,

0:03:30.533,0:03:32.246
to pick players, from one to use

0:03:32.246,0:03:35.068
smart data and rigorous statistical analysis

0:03:35.068,0:03:38.439
to figure out how to pick players[br]that would help them win games.

0:03:38.439,0:03:40.237
It worked for the Oakland A's,

0:03:40.237,0:03:42.456
and it worked in the State of New Jersey.

0:03:42.456,0:03:44.529
We took Camden off the top of the list

0:03:44.529,0:03:46.700
as the most dangerous city in America.

0:03:46.700,0:03:49.855
We reduced murders there by 41 percent,

0:03:49.855,0:03:52.837
which actually means 37 lives were saved.

0:03:52.837,0:03:56.577
And we reduced all crime in the city by 26 percent.

0:03:56.577,0:03:59.816
We also changed the way[br]we did criminal prosecutions.

0:03:59.816,0:04:01.821
So we went from doing low-level drug crimes

0:04:01.821,0:04:03.463
that were outside our building

0:04:03.463,0:04:05.805
to doing cases of state-wide importance,

0:04:05.805,0:04:08.963
on things like reducing violence[br]with the most violent offenders,

0:04:08.963,0:04:10.821
prosecuting street gangs,

0:04:10.821,0:04:14.229
gun and drug trafficking, and political corruption.

0:04:14.229,0:04:16.731
And all of this matters greatly,

0:04:16.731,0:04:18.676
because public safety to me

0:04:18.676,0:04:21.212
is the most important function of government.

0:04:21.212,0:04:23.510
If we're not safe, we can't be educated,

0:04:23.510,0:04:24.858
we can't be healthy,

0:04:24.858,0:04:25.738
we can't do any of the other things

0:04:25.738,0:04:28.327
we want to do in our lives.

0:04:28.327,0:04:29.504
And we live in a country today

0:04:29.504,0:04:32.638
where we face serious criminal justice problems.

0:04:32.638,0:04:36.299
We have 12 million arrests every single year.

0:04:36.299,0:04:38.342
The vast majority of those arrests

0:04:38.342,0:04:41.170
are for low-level crimes, like misdemeanors,

0:04:41.170,0:04:43.088
70 to 80 percent.

0:04:43.088,0:04:45.079
Less than five percent of all arrests

0:04:45.079,0:04:46.974
are for violent crime.

0:04:46.974,0:04:49.029
Yet we spend 75 billion,

0:04:49.029,0:04:50.447
that's b for billion,

0:04:50.447,0:04:54.574
dollars a year on state and local corrections costs.

0:04:54.574,0:04:57.415
Right now, today, we have 2.3 million people

0:04:57.415,0:04:59.315
in our jails and prisons.

0:04:59.315,0:05:02.111
And we face unbelievable public safety challenges

0:05:02.111,0:05:04.050
because we have a situation

0:05:04.050,0:05:06.625
in which two thirds of the people in our jails

0:05:06.625,0:05:08.702
are there waiting for trial.

0:05:08.702,0:05:10.837
They haven't yet been convicted of a crime.

0:05:10.837,0:05:12.956
They're just waiting for their day in court.

0:05:12.956,0:05:16.504
And 67 percent of people come back.

0:05:16.504,0:05:18.471
Our recidivism rate is amongst the highest

0:05:18.471,0:05:19.526
in the world.

0:05:19.526,0:05:21.359
Almost seven in 10 people who are released

0:05:21.359,0:05:23.286
from prison will be rearrested

0:05:23.286,0:05:27.241
in a constant cycle of crime and incarceration.

0:05:27.241,0:05:29.823
So when I started my job at the Arnold Foundation,

0:05:29.823,0:05:32.559
I came back to looking at a lot of these questions,

0:05:32.559,0:05:34.213
and I came back to thinking about how

0:05:34.213,0:05:36.304
we had used data and analytics to transform

0:05:36.304,0:05:39.180
the way we did criminal justice in New Jersey.

0:05:39.180,0:05:41.063
And when I look at the criminal justice system

0:05:41.063,0:05:42.780
in the United States today,

0:05:42.780,0:05:44.466
I feel the exact same way that I did

0:05:44.466,0:05:47.085
about the State of New Jersey when I started there,

0:05:47.085,0:05:50.313
which is that we absolutely have to do better,

0:05:50.313,0:05:52.560
and I know that we can do better.

0:05:52.560,0:05:53.941
So I decided to focus

0:05:53.941,0:05:55.835
on using data and analytics

0:05:55.835,0:05:58.519
to help make the most critical decision

0:05:58.519,0:06:00.125
in public safety,

0:06:00.125,0:06:02.146
and that decision is the determination

0:06:02.146,0:06:04.681
of whether, when someone has been arrested,

0:06:04.681,0:06:06.596
whether they pose a risk to public safety

0:06:06.596,0:06:08.122
and should be detained,

0:06:08.122,0:06:10.478
or whether they don't pose a risk to public safety

0:06:10.478,0:06:12.115
and should be released.

0:06:12.115,0:06:13.881
Everything that happens in criminal cases

0:06:13.881,0:06:15.806
comes out of this one decision.

0:06:15.806,0:06:17.302
It impacts everything.

0:06:17.302,0:06:18.652
It impacts sentencing.

0:06:18.652,0:06:20.677
It impacts whether someone gets drug treatment.

0:06:20.677,0:06:22.876
It impacts crime and violence.

0:06:22.876,0:06:24.813
And when I talk to judges around the United States,

0:06:24.813,0:06:26.772
which I do all the time now,

0:06:26.772,0:06:28.578
they all say the same thing,

0:06:28.578,0:06:31.685
which is that we put dangerous people in jail,

0:06:31.685,0:06:35.210
and we let non-dangerous, non-violent people out.

0:06:35.210,0:06:37.443
They mean it and they believe it.

0:06:37.443,0:06:39.176
But when you start to look at the data,

0:06:39.176,0:06:41.640
which, by the way, the judges don't have,

0:06:41.640,0:06:43.252
when we start to look at the data,

0:06:43.252,0:06:45.486
what we find time and time again,

0:06:45.486,0:06:47.652
is that this isn't the case.

0:06:47.652,0:06:49.333
We find low-risk offenders,

0:06:49.333,0:06:53.047
which makes up 50 percent of our[br]entire criminal justice population,

0:06:53.047,0:06:55.446
we find that they're in jail.

0:06:55.446,0:06:57.932
Take Leslie Chew, who was a Texas man

0:06:57.932,0:07:00.816
who stole four blankets on a cold winter night.

0:07:00.816,0:07:03.411
He was arrested, and he was kept in jail

0:07:03.411,0:07:05.464
on 3,500 dollars bail,

0:07:05.464,0:07:08.240
an amount that he could not afford to pay.

0:07:08.240,0:07:10.828
And he stayed in jail for eight months

0:07:10.828,0:07:12.893
until his case came up for trial,

0:07:12.893,0:07:16.798
at a cost to taxpayers of more than 9,000 dollars.

0:07:16.798,0:07:18.626
And at the other end of the spectrum,

0:07:18.626,0:07:21.077
we're doing an equally terrible job.

0:07:21.077,0:07:22.649
The people who we find

0:07:22.649,0:07:24.668
are the highest risk offenders,

0:07:24.668,0:07:26.689
the people who we think have the highest likelihood

0:07:26.689,0:07:29.117
of committing a new crime if they're released,

0:07:29.117,0:07:31.929
we see nationally that 50 percent of those people

0:07:31.929,0:07:34.041
are being released.

0:07:34.041,0:07:37.215
The reason for this is the way we make decisions.

0:07:37.215,0:07:39.740
Judges have the best intentions

0:07:39.740,0:07:41.970
when they make these decisions about risk,

0:07:41.970,0:07:43.360
but they're making them subjectively.

0:07:43.360,0:07:45.583
They're like the baseball scouts 20 years ago

0:07:45.583,0:07:47.637
who were using their instinct and their experience

0:07:47.637,0:07:50.871
to try to decide what risk someone poses.

0:07:50.871,0:07:52.125
They're being subjective,

0:07:52.125,0:07:54.937
and we know what happens[br]with subjective decision-making,

0:07:54.937,0:07:57.266
which is that we are often wrong.

0:07:57.266,0:07:59.202
What we need in this space

0:07:59.202,0:08:01.800
are strong data and analytics.

0:08:01.800,0:08:03.331
What I decided to look for

0:08:03.331,0:08:06.321
was a strong data and analytic risk assessment tool,

0:08:06.321,0:08:08.870
something that would let judges actually understand

0:08:08.870,0:08:11.314
with a scientific and objective way

0:08:11.314,0:08:12.930
what the risk was that was posed

0:08:12.930,0:08:14.663
by someone in front of them.

0:08:14.663,0:08:16.266
I looked all over the country,

0:08:16.266,0:08:18.470
and I found that between five and 10 percent

0:08:18.470,0:08:19.537
of all U.S. jurisdictions

0:08:19.537,0:08:22.393
actually use any type of risk assessment tool,

0:08:22.393,0:08:23.940
and when I looked at these tools,

0:08:23.940,0:08:25.892
I quickly realized why.

0:08:25.892,0:08:28.367
They were unbelievably expensive to administer,

0:08:28.367,0:08:30.110
they were time-consuming,

0:08:30.110,0:08:32.680
they were limited to the local jurisdiction

0:08:32.680,0:08:33.739
in which they'd been created.

0:08:33.739,0:08:35.287
So basically, they couldn't be scaled

0:08:35.287,0:08:37.726
or transferred to other places.

0:08:37.726,0:08:39.901
So I went out and build a phenomenal team

0:08:39.901,0:08:42.900
of data scientists and researchers

0:08:42.900,0:08:43.587
and statisticians

0:08:43.587,0:08:46.339
to build a universal risk assessment tool,

0:08:46.339,0:08:48.732
so that every single judge in[br]the United States of America

0:08:48.732,0:08:53.380
can have an objective, scientific measure of risk.

0:08:53.380,0:08:54.714
In the tool that we've built,

0:08:54.714,0:08:57.382
what we did was we collected 1.5 million cases

0:08:57.382,0:08:59.465
from all around the United States,

0:08:59.465,0:09:01.710
from cities, from counties,

0:09:01.710,0:09:02.635
from every single state in the country,

0:09:02.635,0:09:04.320
the federal districts.

0:09:04.320,0:09:06.370
And with those 1.5 million cases,

0:09:06.370,0:09:08.310
which is the largest data set on pretrial

0:09:08.310,0:09:10.870
in the United States today,

0:09:10.870,0:09:12.273
we were able to basically find that there were

0:09:12.273,0:09:15.302
900-plus risk factors that we could look at

0:09:15.302,0:09:18.800
to try to figure out what mattered most.

0:09:18.800,0:09:20.466
And we found that there were nine specific things

0:09:20.466,0:09:22.484
that mattered all across the country

0:09:22.484,0:09:25.478
and that were the most highly predictive of risk.

0:09:25.478,0:09:28.982
And so we built a universal risk assessment tool.

0:09:28.982,0:09:30.827
And it looks like this.

0:09:30.827,0:09:33.223
As you'll see, we put some information in,

0:09:33.223,0:09:35.390
but most of it is incredibly simple,

0:09:35.390,0:09:36.992
it's easy to use,

0:09:36.992,0:09:39.761
it focuses on things like the[br]defendant's prior convictions,

0:09:39.761,0:09:41.786
whether they've been sentenced to incarceration,

0:09:41.786,0:09:43.957
whether they've engaged in violence before,

0:09:43.957,0:09:46.397
whether they've even failed to come back to court.

0:09:46.397,0:09:48.912
And with this tool, we can predict three things.

0:09:48.912,0:09:50.596
First, whether or not someone will commit

0:09:50.596,0:09:52.315
a new crime if they're released.

0:09:52.315,0:09:53.979
Second, for the first time,

0:09:53.979,0:09:55.840
and I think this is incredibly important,

0:09:55.840,0:09:57.517
we can predict whether someone will commit

0:09:57.517,0:09:59.689
an act of violence if they're released.

0:09:59.689,0:10:01.391
And that's the single most important thing

0:10:01.391,0:10:03.244
that judges say when you talk to them.

0:10:03.244,0:10:04.889
And third, we can predict whether someone

0:10:04.889,0:10:06.802
will come back to court.

0:10:06.802,0:10:10.680
And every single judge in the[br]United States of America can use it,

0:10:10.680,0:10:13.554
because it's been created on a universal data set.

0:10:13.554,0:10:15.994
What judges see if they run the risk assessment tool

0:10:15.994,0:10:18.559
is this: it's a dashboard.

0:10:18.559,0:10:21.500
At the top, you see the new criminal activity score,

0:10:21.500,0:10:23.860
six of course being the highest,

0:10:23.860,0:10:26.970
and then in the middle you[br]see "elevated risk of violence."

0:10:26.970,0:10:27.793
What that says is that this person

0:10:27.793,0:10:29.655
is someone who has an elevated risk of violence

0:10:29.655,0:10:31.584
that the judge should look twice at.

0:10:31.584,0:10:32.813
And then, towards the bottom,

0:10:32.813,0:10:34.734
you see the "Failure to Appear" score,

0:10:34.734,0:10:36.850
which again is the likelihood

0:10:36.850,0:10:38.787
that someone will come back to court.

0:10:38.787,0:10:41.537
Now I want to say something really important.

0:10:41.537,0:10:43.865
It's not that I think we should be eliminating

0:10:43.865,0:10:46.416
the judge's instinct and experience

0:10:46.416,0:10:48.112
from this process.

0:10:48.112,0:10:49.294
I don't.

0:10:49.294,0:10:51.900
I actually believe the problem that we see

0:10:51.900,0:10:53.940
and the reason that we have[br]these incredible system errors,

0:10:53.940,0:10:56.933
where we're incarcerating[br]low-level, nonviolent people

0:10:56.933,0:11:00.229
and we're releasing high-risk, dangerous people,

0:11:00.229,0:11:02.998
is that we don't have an objective measure of risk.

0:11:02.998,0:11:04.313
But what I believe should happen

0:11:04.313,0:11:06.867
is that we should take that[br]data-driven risk assessment

0:11:06.867,0:11:09.908
and combine that with the[br]judge's instinct and experience

0:11:09.908,0:11:12.897
to lead us to better decision-making.

0:11:12.897,0:11:16.415
The tool went state-wide in Kentucky on July 1st,

0:11:16.415,0:11:19.720
and we're about to go up in a[br]number of other U.S. jurisdictions.

0:11:19.720,0:11:22.220
Our goal, quite simply, is that every single judge

0:11:22.220,0:11:24.472
in the United States will use a data-driven risk tool

0:11:24.472,0:11:26.548
within the next five years.

0:11:26.548,0:11:28.900
We're now working on risk tools

0:11:28.900,0:11:31.230
for prosecutors and for police officers, as well,

0:11:31.230,0:11:33.762
to try to take a system that runs today

0:11:33.762,0:11:36.772
in America the same way it did 50 years ago,

0:11:36.772,0:11:38.762
based on instinct and experience,

0:11:38.762,0:11:40.586
and make it into one that runs

0:11:40.586,0:11:43.286
on data and analytics.

0:11:43.286,0:11:45.930
Now, the great news about all this,

0:11:45.930,0:11:46.762
and we have a ton of work left to do,

0:11:46.762,0:11:48.527
and we have a lot of culture to change,

0:11:48.527,0:11:50.366
but the great news about all of it

0:11:50.366,0:11:52.710
is that we know it works.

0:11:52.710,0:11:54.740
It's why Google is Google,

0:11:54.740,0:11:56.802
and it's why all these baseball teams use moneyball

0:11:56.802,0:11:58.598
to win games.

0:11:58.598,0:12:00.350
The great news for us as well

0:12:00.350,0:12:02.123
is that it's the way that we can transform

0:12:02.123,0:12:04.492
the American criminal justice system.

0:12:04.492,0:12:06.924
It's how we can make our streets safer,

0:12:06.924,0:12:09.470
we can reduce our prison costs,

0:12:09.470,0:12:11.290
and we can make our system much fairer

0:12:11.290,0:12:12.969
and more just.

0:12:12.969,0:12:14.870
Some people call it data science.

0:12:14.870,0:12:17.356
I call it moneyballing criminal justice.

0:12:17.356,0:12:19.175
Thank you.

0:12:19.175,0:12:23.175
(Applause)