0:00:00.984,0:00:02.275
My job at Twitter

0:00:02.275,0:00:04.253
is to ensure user trust,

0:00:04.253,0:00:07.090
protect user rights and keep users safe,

0:00:07.090,0:00:08.350
both from each other

0:00:08.350,0:00:12.249
and, at times, from themselves.

0:00:12.249,0:00:16.524
Let's talk about what scale looks like at Twitter.

0:00:16.524,0:00:19.394
Back in January 2009,

0:00:19.394,0:00:22.725
we saw more than two million new tweets each day

0:00:22.725,0:00:24.489
on the platform.

0:00:24.489,0:00:30.397
January 2014, more than 500 million.

0:00:30.397,0:00:32.889
We were seeing two million tweets

0:00:32.889,0:00:35.065
in less than six minutes.

0:00:35.065,0:00:42.049
That's a 24,900-percent increase.

0:00:42.049,0:00:45.302
Now, the vast majority of activity on Twitter

0:00:45.302,0:00:46.805
puts no one in harm's way.

0:00:46.805,0:00:48.740
There's no risk involved.

0:00:48.740,0:00:54.493
My job is to root out and prevent activity that might.

0:00:54.493,0:00:56.466
Sounds straightforward, right?

0:00:56.466,0:00:57.618
You might even think it'd be easy,

0:00:57.618,0:00:59.788
given that I just said the vast majority

0:00:59.788,0:01:03.598
of activity on Twitter puts no one in harm's way.

0:01:03.598,0:01:05.767
Why spend so much time

0:01:05.767,0:01:08.510
searching for potential calamities

0:01:08.510,0:01:11.410
in innocuous activities?

0:01:11.410,0:01:14.350
Given the scale that Twitter is at,

0:01:14.350,0:01:16.707
a one-in-a-million chance happens

0:01:16.707,0:01:21.583
500 times a day.

0:01:21.583,0:01:23.028
It's the same for other companies

0:01:23.028,0:01:24.499
dealing at this sort of scale.

0:01:24.499,0:01:26.207
For us, edge cases,

0:01:26.207,0:01:29.832
those rare situations that are unlikely to occur,

0:01:29.832,0:01:32.454
are more like norms.

0:01:32.454,0:01:36.396
Say 99.999 percent of tweets

0:01:36.396,0:01:38.284
pose no risk to anyone.

0:01:38.284,0:01:39.350
There's no threat involved.

0:01:39.350,0:01:42.304
Maybe people are documenting travel landmarks

0:01:42.304,0:01:44.267
like Australia's Heart Reef,

0:01:44.267,0:01:47.188
or tweeting about a concert they're attending,

0:01:47.188,0:01:51.935
or sharing pictures of cute baby animals.

0:01:51.935,0:01:56.444
After you take out that 99.999 percent,

0:01:56.444,0:01:59.973
that tiny percentage of tweets remaining

0:01:59.973,0:02:02.362
works out to roughly

0:02:02.362,0:02:05.837
150,000 per month.

0:02:05.837,0:02:08.293
The sheer scale of what we're dealing with

0:02:08.293,0:02:10.605
makes for a challenge.

0:02:10.605,0:02:11.783
You know what else makes my role

0:02:11.783,0:02:14.890
particularly challenging?

0:02:14.890,0:02:20.013
People do weird things.

0:02:20.013,0:02:21.842
(Laughter)

0:02:21.842,0:02:24.233
And I have to figure out what they're doing,

0:02:24.233,0:02:26.482
why, and whether or not there's risk involved,

0:02:26.482,0:02:28.650
often without much in terms of context

0:02:28.650,0:02:30.497
or background.

0:02:30.497,0:02:32.574
I'm going to show you some examples

0:02:32.574,0:02:34.579
that I've run into during my time at Twitter --

0:02:34.579,0:02:36.199
these are all real examples —

0:02:36.199,0:02:38.852
of situations that at first seemed cut and dried,

0:02:38.852,0:02:40.495
but the truth of the matter was something

0:02:40.495,0:02:42.045
altogether different.

0:02:42.045,0:02:44.022
The details have been changed

0:02:44.022,0:02:45.279
to protect the innocent

0:02:45.279,0:02:48.512
and sometimes the guilty.

0:02:48.512,0:02:51.517
We'll start off easy.

0:02:51.517,0:02:53.310
["Yo bitch"]

0:02:53.310,0:02:56.538
If you saw a Tweet that only said this,

0:02:56.538,0:02:58.232
you might think to yourself,

0:02:58.232,0:02:59.885
"That looks like abuse."

0:02:59.885,0:03:02.992
After all, why would you[br]want to receive the message,

0:03:02.992,0:03:05.210
"Yo, bitch."

0:03:05.210,0:03:09.873
Now, I try to stay relatively hip

0:03:09.873,0:03:12.385
to the latest trends and memes,

0:03:12.385,0:03:15.089
so I knew that "yo, bitch"

0:03:15.089,0:03:18.243
was also often a common greeting between friends,

0:03:18.243,0:03:22.505
as well as being a popular "Breaking Bad" reference.

0:03:22.505,0:03:24.992
I will admit that I did not expect

0:03:24.992,0:03:27.833
to encounter a fourth use case.

0:03:27.833,0:03:30.937
It turns out it is also used on Twitter

0:03:30.937,0:03:33.999
when people are role-playing as dogs.

0:03:33.999,0:03:39.278
(Laughter)

0:03:39.278,0:03:40.944
And in fact, in that case,

0:03:40.944,0:03:42.553
it's not only not abusive,

0:03:42.553,0:03:45.692
it's technically just an accurate greeting.

0:03:45.692,0:03:48.581
(Laughter)

0:03:48.581,0:03:50.652
So okay, determining whether or not

0:03:50.652,0:03:52.500
something is abusive without context,

0:03:52.500,0:03:54.092
definitely hard.

0:03:54.092,0:03:56.809
Let's look at spam.

0:03:56.809,0:03:58.769
Here's an example of an account engaged

0:03:58.769,0:04:00.437
in classic spammer behavior,

0:04:00.437,0:04:01.996
sending the exact same message

0:04:01.996,0:04:03.800
to thousands of people.

0:04:03.800,0:04:06.593
While this is a mockup I put[br]together using my account,

0:04:06.593,0:04:09.594
we see accounts doing this all the time.

0:04:09.594,0:04:11.573
Seems pretty straightforward.

0:04:11.573,0:04:13.626
We should just automatically suspend accounts

0:04:13.626,0:04:16.933
engaging in this kind of behavior.

0:04:16.933,0:04:20.143
Turns out there's some exceptions to that rule.

0:04:20.143,0:04:23.026
Turns out that that message[br]could also be a notification

0:04:23.026,0:04:26.915
you signed up for that the International[br]Space Station is passing overhead

0:04:26.915,0:04:28.761
because you wanted to go outside

0:04:28.761,0:04:30.709
and see if you could see it.

0:04:30.709,0:04:31.934
You're not going to get that chance

0:04:31.934,0:04:33.781
if we mistakenly suspend the account

0:04:33.781,0:04:36.047
thinking it's spam.

0:04:36.047,0:04:39.573
Okay. Let's make the stakes higher.

0:04:39.573,0:04:41.489
Back to my account,

0:04:41.489,0:04:44.994
again exhibiting classic behavior.

0:04:44.994,0:04:47.637
This time it's sending the same message and link.

0:04:47.637,0:04:50.411
This is often indicative of [br]something called phishing,

0:04:50.411,0:04:53.589
somebody trying to steal another[br]person's account information

0:04:53.589,0:04:55.792
by directing them to another website.

0:04:55.792,0:04:59.986
That's pretty clearly not a good thing.

0:04:59.986,0:05:01.916
We want to, and do, suspend accounts

0:05:01.916,0:05:04.540
engaging in that kind of behavior.

0:05:04.540,0:05:07.787
So why are the stakes higher for this?

0:05:07.787,0:05:10.786
Well, this could also be a bystander at a rally

0:05:10.786,0:05:12.696
who managed to record a video

0:05:12.696,0:05:15.966
of a police officer beating a non-violent protester

0:05:15.966,0:05:18.941
who's trying to let the world know what's happening.

0:05:18.941,0:05:20.584
We don't want to gamble

0:05:20.584,0:05:23.101
on potentially silencing that crucial speech

0:05:23.101,0:05:26.030
by classifying it as spam and suspending it.

0:05:26.030,0:05:28.909
That means we evaluate hundreds of parameters

0:05:28.909,0:05:30.597
when looking at account behaviors,

0:05:30.597,0:05:32.613
and even then, we can still get it wrong

0:05:32.613,0:05:34.849
and have to reevaluate.

0:05:34.849,0:05:38.557
Now, given the sorts of challenges I'm up against,

0:05:38.557,0:05:41.253
it's crucial that I not only predict

0:05:41.253,0:05:45.037
but also design protections for the unexpected.

0:05:45.037,0:05:47.379
And that's not just an issue for me,

0:05:47.379,0:05:49.466
or for Twitter, it's an issue for you.

0:05:49.466,0:05:51.872
It's an issue for anybody who's building or creating

0:05:51.872,0:05:53.797
something that you think is going to be amazing

0:05:53.797,0:05:56.586
and will let people do awesome things.

0:05:56.586,0:05:59.452
So what do I do?

0:05:59.452,0:06:02.770
I pause and I think,

0:06:02.770,0:06:04.865
how could all of this

0:06:04.865,0:06:08.658
go horribly wrong?

0:06:08.658,0:06:13.111
I visualize catastrophe.

0:06:13.111,0:06:15.574
And that's hard. There's a sort of

0:06:15.574,0:06:18.422
inherent cognitive dissonance in doing that,

0:06:18.422,0:06:20.234
like when you're writing your wedding vows

0:06:20.234,0:06:22.880
at the same time as your prenuptial agreement.

0:06:22.880,0:06:24.576
(Laughter)

0:06:24.576,0:06:26.949
But you still have to do it,

0:06:26.949,0:06:31.395
particularly if you're marrying [br]500 million tweets per day.

0:06:31.395,0:06:34.492
What do I mean by "visualize catastrophe?"

0:06:34.492,0:06:37.254
I try to think of how something as

0:06:37.254,0:06:40.482
benign and innocuous as a picture of a cat

0:06:40.482,0:06:41.586
could lead to death,

0:06:41.586,0:06:43.912
and what to do to prevent that.

0:06:43.912,0:06:46.295
Which happens to be my next example.

0:06:46.295,0:06:49.405
This is my cat, Eli.

0:06:49.405,0:06:51.386
We wanted to give users the ability

0:06:51.386,0:06:53.459
to add photos to their tweets.

0:06:53.459,0:06:55.056
A picture is worth a thousand words.

0:06:55.056,0:06:57.065
You only get 140 characters.

0:06:57.065,0:06:58.265
You add a photo to your tweet,

0:06:58.265,0:07:01.303
look at how much more content you've got now.

0:07:01.303,0:07:02.980
There's all sorts of great things you can do

0:07:02.980,0:07:04.987
by adding a photo to a tweet.

0:07:04.987,0:07:07.267
My job isn't to think of those.

0:07:07.267,0:07:10.014
It's to think of what could go wrong.

0:07:10.014,0:07:11.906
How could this picture

0:07:11.906,0:07:15.445
lead to my death?

0:07:15.445,0:07:18.605
Well, here's one possibility.

0:07:18.605,0:07:21.691
There's more in that picture than just a cat.

0:07:21.691,0:07:23.783
There's geodata.

0:07:23.783,0:07:25.995
When you take a picture with your smartphone

0:07:25.995,0:07:27.294
or digital camera,

0:07:27.294,0:07:28.948
there's a lot of additional information

0:07:28.948,0:07:30.564
saved along in that image.

0:07:30.564,0:07:32.496
In fact, this image also contains

0:07:32.496,0:07:34.301
the equivalent of this,

0:07:34.301,0:07:37.380
more specifically, this.

0:07:37.380,0:07:39.336
Sure, it's not likely that someone's going to try

0:07:39.336,0:07:41.621
to track me down and do me harm

0:07:41.621,0:07:43.405
based upon image data associated

0:07:43.405,0:07:45.353
with a picture I took of my cat,

0:07:45.353,0:07:49.004
but I start by assuming the worst will happen.

0:07:49.004,0:07:51.342
That's why, when we launched photos on Twitter,

0:07:51.342,0:07:55.163
we made the decision to strip that geodata out.

0:07:55.163,0:08:01.010
(Applause)

0:08:01.010,0:08:03.623
If I start by assuming the worst

0:08:03.623,0:08:04.570
and work backwards,

0:08:04.570,0:08:07.123
I can make sure that the protections we build

0:08:07.123,0:08:08.891
work for both expected

0:08:08.891,0:08:10.969
and unexpected use cases.

0:08:10.969,0:08:13.914
Given that I spend my days and nights

0:08:13.914,0:08:16.455
imagining the worst that could happen,

0:08:16.455,0:08:20.712
it wouldn't be surprising if [br]my worldview was gloomy.

0:08:20.712,0:08:22.495
(Laughter)

0:08:22.495,0:08:23.912
It's not.

0:08:23.912,0:08:27.788
The vast majority of interactions I see --

0:08:27.788,0:08:31.689
and I see a lot, believe me -- are positive,

0:08:31.689,0:08:33.613
people reaching out to help

0:08:33.613,0:08:37.061
or to connect or share information with each other.

0:08:37.061,0:08:40.384
It's just that for those of us dealing with scale,

0:08:40.384,0:08:44.184
for those of us tasked with keeping people safe,

0:08:44.184,0:08:46.730
we have to assume the worst will happen,

0:08:46.730,0:08:50.957
because for us, a one-in-a-million chance

0:08:50.957,0:08:53.706
is pretty good odds.

0:08:53.706,0:08:55.570
Thank you.

0:08:55.570,0:08:59.570
(Applause)