-
My job at Twitter
-
is to ensure user trust,
-
protect user rights, and keep users safe,
-
both from each other
-
and, at times, from themselves.
-
Let's talk about what scale looks like at Twitter.
-
Back in January 2009,
-
we saw more than two million Tweets each day
-
on the platform.
-
January 2014, more than 500 million.
-
We were seeing two million Tweets
-
in less than six minutes.
-
That's a 24,900 percent increase.
-
Now, the vast majority of activity on Twitter
-
puts no one in harm's way.
-
There's no risk involved.
-
My job is to root out and prevent activity that might.
-
Sounds straightforward, right?
-
You might even think it'd be easy,
-
given that I just said the vast majority
-
of activity on Twitter puts no one in harm's way.
-
Why spend so much time
-
searching for potential calamities
-
in innocuous activities?
-
Given the scale that Twitter is at,
-
a one in a million chance happens
-
five hundred times a day.
-
It's the same for other companies
-
dealing at this sort of scale.
-
For us edge cases,
-
those rare situations that are unlikely to occur,
-
are more like norms.
-
Say 99.999 percent of Tweets
-
pose no risk to anyone.
-
There's no threat involved.
-
Maybe people are documenting travel landmarks
-
like Australia's Hard Reef,
-
or Tweeting about a concert they're attending,
-
or sharing pictures of cute baby animals.
-
After you take out that 99.999 percent,
-
that tiny percentage of Tweets remaining
-
works out to roughly
-
150,000 per month.
-
The sheer scale of what we're dealing with
-
makes for a challenge.
-
You know what else makes my role
-
particularly challenging?
-
People do weird things.
-
(Laughter)
-
And I have to figure out what they're doing,
-
why, and whether or not there's risk involved,
-
often without much in terms of context
-
or background.
-
I'm going to show you some examples
-
that I've run into during my time at Twitter
-
— these are all real examples —
-
of situations that at first seemed cut and dried,
-
but the truth of the matter was something
-
altogether different.
-
The details have been changed
-
to protect the innocent
-
and sometimes the guilty.
-
We'll start off easy.
-
[yo bitch]
-
If you saw a Tweet that only said this,
-
you might think to yourself,
-
"That looks like abuse."
-
After all, why would you
want to receive the message,
-
"yo bitch."
-
Now, I try to stay relatively hip
-
to the latest trends and memes,
-
so I knew that "yo bitch"
-
was also often a common greeting between friends,
-
as well as being a popular Breaking Bad reference.
-
I will admit that I did not expect
-
to encounter a fourth use case.
-
It turns out it is also used on Twitter
-
when people are roleplaying as dogs.
-
(Laughter)
-
And in fact, in that case,
-
it's not only not abusive,
-
it's technically just an accurate greeting.
-
(Laughter)
-
So okay, determining whether or not
-
something is abusive without context,
-
definitely hard.
-
Let's look at spam.
-
Here's an example of an account engaged
-
in classic Spammer behavior,
-
sending the exact same message
-
to thousands of people.
-
While this is a mockup I put
together using my account,
-
we see accounts doing this all the time.
-
Seems pretty straightforward.
-
We should just automatically suspend accounts
-
engaging in this kind of behavior.
-
Turns out there's some exceptions to that rule.
-
Turns out that that message
could also be a notification
-
you signed up for that the International
Space Station is passing overhead
-
because you wanted to go outside
-
and see if you could see it.
-
You're not going to get that chance
-
if we mistakenly suspend the account
-
thinking it's spam.
-
Okay. Let's make the stakes higher.
-
Back to my account,
-
again exhibiting classic behavior.
-
This time it's sending the same message and link.
-
This is often indicative of something called phishing,
-
somebody trying to steal another
person's account information
-
by directing them to another website.
-
That's pretty clearly not a good thing.
-
We want to, and do, suspend accounts
-
engaging in that kind of behavior.
-
So why are the stakes higher for this?
-
Well, this could also be a bystander at a rally
-
who managed to record a video
-
of a police officer beating a non-violent protester
-
who's trying to let the world know what's happening.
-
We don't want to gamble
-
on potentially silencing that crucial speech
-
by classifying it as spam and suspending it.
-
That means we evaluate hundreds of parameters
-
when looking at account behaviors,
-
and even then, we can still get it wrong
-
and have to reevaluate.
-
Now, given the sorts of challenges I'm up against,
-
it's crucial that I not only predict
-
but also design protections for the unexpected.
-
And that's not just an issue for me,
-
or for Twitter, it's an issue for you.
-
It's an issue for anybody who's building or creating
-
something that you think is going to be amazing
-
and will let people do awesome things.
-
So what do I do?
-
I pause and I think,
-
how could all of this
-
go horribly wrong?
-
I visualize catastrophe.
-
And that's hard. There's a sort of
-
inherent cognitive dissonance in doing that,
-
like when you're writing your wedding vows
-
at the same time as your prenuptial agreement.
-
(Laughter)
-
But you still have to do it,
-
particularly if you're marrying 500 million Tweets
-
per day.
-
What do I mean by "visualize catastrophe?"
-
I try to think about how something as, say,
-
benign and innocuous as a picture of a cat
-
could lead to death,
-
and what to do to prevent that.
-
Which happens to be my next example.
-
This is my cat, Eli.
-
We wanted to give users the ability
-
to add photos to their Tweets.
-
A picture is worth a thousand words.
-
You only get 140 characters.
-
You add a photo to your Tweet,
-
look at how much more content you've got now.
-
There's all sorts of great things you can do
-
by adding a photo to a Tweet.
-
My job isn't to think of those.
-
It's to think of what could go wrong.
-
How could this picture
-
lead to my death?
-
Well, here's on possibility.
-
There's more in that picture than just a cat.
-
There's geodata.
-
When you take a picture with your smartphone
-
or digital camera,
-
there's a lot of additional information
-
saved along in that image.
-
In fact, this image also contains
-
the equivalent of this,
-
more specifically, this.
-
Sure, it's not likely that someone's going to try
-
to track me down and do me harm
-
based upon image data associated
-
with a picture I took of my cat,
-
but I start by assuming the worst will happen.
-
That's why, when we launched photos on Twitter,
-
we made the decision to strip that geodata out.
-
(Applause)
-
If I start by assuming the worst
-
and work backwards,
-
I can make sure that the protections we build
-
work for both expected
-
and unexpected use cases.
-
Given that I spend my days and nights
-
imagining the worst that could happen,
-
it wouldn't be surprising if my worldview was gloomy.
-
(Laughter)
-
It's not.
-
The vast majority of interactions I see,
-
and I see a lot, believe me, are positive,
-
people reaching out to help
-
or to connect or share information with each other.
-
It's just that for those of dealing with scale,
-
for those of us tasked with keeping people safe,
-
we have to assume the worst will happen,
-
because for us, a one in a million chance
-
is pretty good odds.
-
Thank you.
-
(Applause)