How the Internet will (one day) transform government

The open-source world has learned to deal with a flood of new, oftentimes divergent, ideas using hosting services like GitHub -- so why can’t governments? In this rousing talk Clay Shirky shows how democracies can take a lesson from the Internet, to be not just transparent but also to draw on the knowledge of all their citizens.

I want to talk to you today about something

the open-source programming world can teach democracy,

but before that, a little preamble.

Let's start here.

This is Martha Payne. Martha's a 9-year-old Scot

who lives in the Council of Argyll and Bute.

A couple months ago, Payne started a food blog

called NeverSeconds, and she would take her camera

with her every day to school to document

her school lunches.

Can you spot the vegetable? (Laughter)

And, as sometimes happens,

this blog acquired first dozens of readers,

and then hundreds of readers,

and then thousands of readers, as people tuned in

to watch her rate her school lunches,

including on my favorite category,

"Pieces of hair found in food." (Laughter)

This was a zero day. That's good.

And then two weeks ago yesterday, she posted this.

A post that read: "Goodbye."

And she said, "I'm very sorry to tell you this, but

my head teacher pulled me out of class today and told me

I'm not allowed to take pictures in the lunch room anymore.

I really enjoyed doing this.

Thank you for reading. Goodbye."

You can guess what happened next, right? (Laughter)

The outrage was so swift, so voluminous, so unanimous,

that the Council of Argyll and Bute reversed themselves

the same day and said, "We would,

we would never censor a nine-year-old." (Laughter)

Except, of course, this morning. (Laughter)

And this brings up the question,

what made them think they could get away

with something like that? (Laughter)

And the answer is, all of human history prior to now.

(Laughter) So,

what happens when a medium suddenly puts

a lot of new ideas into circulation?

Now, this isn't just a contemporaneous question.

This is something we've faced several times

over the last few centuries.

When the telegraph came along, it was clear

that it was going to globalize the news industry.

What would this lead to?

Well, obviously, it would lead to world peace.

The television, a medium that allowed us not just to hear

but see, literally see, what was going on

elsewhere in the world, what would this lead to?

World peace. (Laughter)

The telephone?

You guessed it: world peace.

Sorry for the spoiler alert, but no world peace. Not yet.

Even the prInting press, even the printing press

was assumed to be a tool that was going to enforce

Catholic intellectual hegemony across Europe.

Instead, what we got was Martin Luther's 95 Theses,

the Protestant Reformation, and, you know,

the Thirty Years' War. All right,

so what all of these predictions of world peace got right

is that when a lot of new ideas suddenly

come into circulation, it changes society.

What they got exactly wrong was what happens next.

The more ideas there are in circulation,

the more ideas there are for any individual to disagree with.

More media always means more arguing.

That's what happens when the media's space expands.

And yet, when we look back on the printing press

in the early years, we like what happened.

We are a pro-printing press society.

So how do we square those two things,

that it leads to more arguing, but we think it was good?

And the answer, I think, can be found in things like this.

This is the cover of "Philosophical Transactions,"

the first scientific journal ever published in English

in the middle of the 1600s,

and it was created by a group of people who had been

calling themselves "The Invisible College,"

a group of natural philosophers who only later

would call themselves scientists,

and they wanted to improve the way

natural philosophers argued with each other,

and they needed to do two things for this.

They needed openness. They needed to create a norm

which said, when you do an experiment,

you have to publish not just your claims,

but how you did the experiment.

If you don't tell us how you did it, we won't trust you.

But the other thing they needed was speed.

They had to quickly synchronize what

other natural philosophers knew. Otherwise,

you couldn't get the right kind of argument going.

The printing press was clearly the right medium for this,

but the book was the wrong tool. It was too slow.

And so they invented the scientific journal

as a way of synchronizing the argument

across the community of natural scientists.

The scientific revolution wasn't created by the printing press.

It was created by scientists,

but it couldn't have been created if they didn't have

a printing press as a tool.

So what about us? What about our generation,

and our media revolution, the Internet?

Well, predictions of world peace? Check. (Laughter)

More arguing? Gold star on that one. (Laughter)

(Laughter)

I mean, YouTube is just a gold mine. (Laughter)

Better arguing? That's the question.

So I study social media, which means,

to a first approximation, I watch people argue.

And if I had to pick a group that I think is

our Invisible College, is our generation's collection of people

trying to take these tools and to press it into service,

not for more arguments, but for better arguments,

I'd pick the open-source programmers.

Programming is a three-way relationship

between a programmer, some source code,

and the computer it's meant to run on, but computers

are such famously inflexible interpreters of instructions

that it's extraordinarily difficult to write out a set

of instructions that the computer knows how to execute,

and that's if one person is writing it.

Once you get more than one person writing it,

it's very easy for any two programmers to overwrite

each other's work if they're working on the same file,

or to send incompatible instructions

that simply causes the computer to choke,

and this problem grows larger

the more programmers are involved.

To a first approximation, the problem of managing

a large software project is the problem

of keeping this social chaos at bay.

Now, for decades there has been a canonical solution

to this problem, which is to use something called

a "version control system,"

and a version control system does what is says on the tin.

It provides a canonical copy of the software

on a server somewhere.

The only programmers who can change it are people

who've specifically been given permission to access it,

and they're only allowed to access the sub-section of it

that they have permission to change.

And when people draw diagrams of version control systems,

the diagrams always look something like this.

All right. They look like org charts.

And you don't have to squint very hard

to see the political ramifications of a system like this.

This is feudalism: one owner, many workers.

Now, that's fine for the commercial software industry.

It really is Microsoft's Office. It's Adobe's Photoshop.

The corporation owns the software.

The programmers come and go.

But there was one programmer who decided

that this wasn't the way to work.

This is Linus Torvalds.

Torvalds is the most famous open-source programmer,

created Linux, obviously, and Torvalds looked at the way

the open-source movement had been dealing with this problem.

Open-source software, the core promise of the open-source license,

is that everybody should have access to all the source code

all the time, but of course, this creates

the very threat of chaos you have to forestall

in order to get anything working.

So most open-source projects just held their noses

and adopted the feudal management systems.

But Torvalds said, "No, I'm not going to do that."

His point of view on this was very clear.

When you adopt a tool, you also adopt

the management philosophy embedded in that tool,

and he wasn't going to adopt anything that didn't work

the way the Linux community worked.

And to give you a sense of how enormous

a decision like this was, this is a map

of the internal dependencies within Linux,

within the Linux operating system, which sub-parts

of the program rely on which other sub-parts to get going.

This is a tremendously complicated process.

This is a tremendously complicated program,

and yet, for years, Torvalds ran this

not with automated tools but out of his email box.

People would literally mail him changes

that they'd agreed on, and he would merge them by hand.

And then, 15 years after looking at Linux and figuring out

how the community worked, he said, "I think I know

how to write a version control system for free people."

And he called it "Git." Git is distributed version control.

It has two big differences

with traditional version control systems.

The first is that it lives up to the philosophical promise

of open-source. Everybody who works on a project

has access to all of the source code all of the time.

And when people draw diagrams of Git workflow,

they use drawings that look like this.

And you don't have to understand what the circles

and boxes and arrows mean to see that this is a far more

complicated way of working than is supported

by ordinary version control systems.

But this is also the thing that brings the chaos back,

and this is Git's second big innovation.

This is a screenshot from GitHub, the premier Git hosting service,

and every time a programmer uses Git

to make any important change at all,

creating a new file, modifying an existing one,

merging two files, Git creates this kind of signature.

This long string of numbers and letters here

is a unique identifier tied to every single change,

but without any central coordination.

Every Git system generates this number the same way,

which means this is a signature tied directly

and unforgeably to a particular change.

This has the following effect:

A programmer in Edinburgh and a programmer in Entebbe

can both get the same -- a copy of the same piece of software.

Each of them can make changes and they can merge them

after the fact even if they didn't know

of each other's existence beforehand.

This is cooperation without coordination.

This is the big change.

Now, I tell you all of this not to convince you that it's great

that open-source programmers now have a tool

that supports their philosophical way of working,

although I think that is great.

I tell you all of this because of what I think it means

for the way communities come together.

Once Git allowed for cooperation without coordination,

you start to see communities form

that are enormously large and complex.

This is a graph of the Ruby community.

It's an open-source programming language,

and all of the interconnections between the people --

this is now not a software graph, but a people graph,

all of the interconnections among the people

working on that project —

and this doesn't look like an org chart.

This looks like a dis-org chart, and yet,

out of this community, but using these tools,

they can now create something together.

So there are two good reasons to think that

this kind of technique can be applied

to democracies in general and in particular to the law.

When you make the claim, in fact,

that something on the Internet is going to be good

for democracy, you often get this reaction.

(Music) (Laughter)

Which is, are you talking about the thing

with the singing cats? Like, is that the thing

you think is going to be good for society?

To which I have to say, here's the thing

with the singing cats. That always happens.

And I don't just mean that always happens with the Internet,

I mean that always happens with media, full stop.

It did not take long after the rise

of the commercial printing press before someone

figured out that erotic novels were a good idea. (Laughter)

You don't have to have an economic incentive to sell books

very long before someone says, "Hey, you know what I bet

people would pay for?" (Laughter)

It took people another 150 years to even think

of the scientific journal, right? So -- (Laughter) (Applause)

So the harnessing by the Invisible College

of the printing press to create the scientific journal

was phenomenally important, but it didn't happen big,

and it didn't happen quick, and it didn't happen fast, so

if you're going to look for where the change is happening,

you have to look on the margins.

So, the law is also dependency-related.

This is a graph of the U.S. Tax Code,

and the dependencies of one law on other laws

for the overall effect.

So there's that as a site for source code management.

But there's also the fact that law is another place

where there are many opinions in circulation,

but they need to be resolved to one canonical copy,

and when you go onto GitHub, and you look around,

there are millions and millions of projects,

almost all of which are source code,

but if you look around the edges, you can see people

experimenting with the political ramifications

of a system like that.

Someone put up all the Wikileaked cables

from the State Department, along with software used

to interpret them, including my favorite use ever

of the Cablegate cables, which is a tool for detecting

naturally occurring haiku in State Department prose.

(Laughter)

Right. (Laughter)

The New York Senate has put up something called

Open Legislation, also hosting it on GitHub,

again for all of the reasons of updating and fluidity.

You can go and pick your Senator and then you can see

a list of bills they have sponsored.

Someone going by Divegeek has put up the Utah code,

the laws of the state of Utah, and they've put it up there

not just to distribute the code,

but with the very interesting possibility that this could

be used to further the development of legislation.

Somebody put up a tool during the copyright debate

last year in the Senate, saying, "It's strange that Hollywood

has more access to Canadian legislators

than Canadian citizens do. Why don't we use GitHub

to show them what a citizen-developed bill might look like?"

And it includes this very evocative screenshot.

This is a called a "diff," this thing on the right here.

This shows you, for text that many people are editing,

when a change was made, who made it,

and what the change is.

The stuff in red is the stuff that got deleted.

The stuff in green is the stuff that got added.

Programmers take this capability for granted.

No democracy anywhere in the world offers this feature

to its citizens for either legislation or for budgets,

even though those are the things done

with our consent and with our money.

Now, I would love to tell you that the fact

that the open-source programmers have worked out

a collaborative method that is large scale, distributed,

cheap, and in sync with the ideals of democracy, I would love

to tell you that because those tools are in place,

the innovation is inevitable. But it's not.

Part of the problem, of course, is just a lack of information.

Somebody put a question up on Quora saying,

"Why is it that lawmakers don't use

distributed version control?"

This, graphically, was the answer. (Laughter)

(Laughter) (Applause)

And that is indeed part of the problem, but only part.

The bigger problem, of course, is power.

The people experimenting with participation don't have

legislative power, and the people who have legislative

power are not experimenting with participation.

They are experimenting with openness.

There's no democracy worth the name that doesn't have

a transparency move, but transparency is openness

in only one direction, and being given a dashboard

without a steering wheel has never been the core promise

a democracy makes to its citizens.

So consider this.

The thing that got Martha Payne's opinions

out into the public was a piece of technology,

but the thing that kept them there was political will.

It was the expectation of the citizens

that she would not be censored.

That's now the state we're in with these collaboration tools.

We have them. We've seen them. They work.

Can we use them?

Can we apply the techniques that worked here to this?

T.S. Eliot once said, "One of the most momentous things

that can happen to a culture

is that they acquire a new form of prose."

I think that's wrong, but -- (Laughter)

I think it's right for argumentation. Right?

A momentous thing that can happen to a culture

is they can acquire a new style of arguing:

trial by jury, voting, peer review, now this. Right?

A new form of arguing has been invented in our lifetimes,

in the last decade, in fact.

It's large, it's distributed, it's low-cost,

and it's compatible with the ideals of democracy.

The question for us now is, are we going to let

the programmers keep it to themselves?

Or are we going to try and take it and press it into service

for society at large?

Thank you for listening. (Applause)

(Applause)

Thank you. Thank you. (Applause)