How I built an information time machine

Imagine if you could surf Facebook ... from the Middle Ages. Well, it may not be as far off as it sounds. In a fun and interesting talk, researcher and engineer Frederic Kaplan shows off the Venice Time Machine, a project to digitize 80 kilometers of books to create a historical and geographical simulation of Venice across 1000 years. (Filmed at TEDxCaFoscariU.)

This is an image of the planet Earth.

It looks very much like the Apollo pictures

that are very well known.

There is something different;

you can click on it,

and if you click on it,

you can zoom in on almost any place on the Earth.

For instance, this is a bird's-eye view

of the EPFL campus.

In many cases, you can also see

how a building looks from a nearby street.

This is pretty amazing.

But there's something missing in this wonderful tour:

It's time.

i'm not really sure when this picture was taken.

I'm not even sure it was taken

at the same moment as the bird's-eye view.

In my lab, we develop tools

to travel not only in space

but also through time.

The kind of question we're asking is

Is it possible to build something

like Google Maps of the past?

Can I add a slider on top of Google Maps

and just change the year,

seeing how it was 100 years before,

1,000 years before?

Is that possible?

Can I reconstruct social networks of the past?

Can I make a Facebook of the Middle Ages?

So, can I build time machines?

Maybe we can just say, "No, it's not possible."

Or, maybe, we can think of it from an information point of view.

This is what I call the information mushroom.

Vertically, you have the time.

and horizontally, the amount of digital information available.

Obviously, in the last 10 years, we have much information.

And obviously the more we go in the past, the less information we have.

If we want to build something like Google Maps of the past,

or Facebook of the past,

we need to enlarge this space,

we need to make that like a rectangle.

How do we do that?

One way is digitization.

There's a lot of material available --

newspaper, printed books, thousands of printed books.

I can digitize all these.

I can extract information from these.

Of course, the more you go in the past,
the less information you will have.

So, it might not be enough.

So, I can do what historians do.

I can extrapolate.

This is what we call, in computer science, simulation.

If I take a log book,

I can consider, it's not just a log book

of a Venetian captain going to a particular journey.

I can consider it is actually a log book

which is representative of
many journeys of that period.

I'm extrapolating.

If I have a painting of a facade,

I can consider it's not just that particular building,

but probably it also shares the same grammar

of buildings where we lost any information.

So if we want to construct a time machine,

we need two things.

We need very large archives,

and we need excellent specialists.

The Venice Time Machine,

the project I'm going to talk to you about,

is a joint project between the EPFL

and the University of Venice Ca'Foscari.

There's something very peculiar about Venice,

that its administration has been

very, very bureaucratic.

They've been keeping track of everything,

almost like Google today.

At the Archivio di Stato,

you have 80 kilometers of archives

documenting every aspect

of the life of Venice over
more than 1,000 years.

You have every boat that goes out,

every boat that comes in.

You have every change that was made in the city.

This is all there.

We are setting up a 10-year digitization program

which has the objective of transforming

this immense archive

into a giant information system.

The type of objective we want to reach

is 450 books a day that can be digitized.

Of course, when you digitize, that's not enough,

because these documents,

most of them are in Latin, in Tuscan,

in Venetian dialect,

so you need to transcribe them,

to translate them in some cases,

to index them,

and this is obviously not easy.

In particular, traditional optical
character recognition method

that can be used for printed manuscripts,

they do not work well on the handwritten document.

So the solution is actually to take inspiration

from another domain: speech recognition.

This is a domain of something
that seems impossible,

which can actually be done,

simply by putting additional constraints.

If you have a very good model

of a language which is used,

if you have a very good model of a document,

how well they are structured.

And these are administrative documents.

They are well structured in many cases.

If you divide this huge archive into smaller subsets

where a smaller subset
actually shares similar features,

then there's a chance of success.

If we reach that stage, then there's something else:

we can extract from this document events.

Actually probably 10 billion events

can be extracted from this archive.

And this giant information system

can be searched in many ways.

You can ask questions like,

"Who lived in this palazzo in 1323?"

"How much cost a sea bream at the Realto market

in 1434?"

"What was the salary

of a glass maker in Murano

maybe over a decade?"

You can ask even bigger questions

because it will be semantically coded.

And then what you can do is put that in space,

because much of this information is spatial.

And from that, you can do things like

reconstructing this extraordinary journey

of that city that managed to
have a sustainable development

over a thousand years,

managing to have all the time

a form of equilibrium with its environment.

You can reconstruct that journey,

visualize it in many different ways.

But of course, you cannot understand
Venice if you just look at the city.

You have to put it in a larger European context.

So the idea is also to document all the things

that worked at the European level.

We can reconstruct also the journey

of the Venetian maritime empire,

how it progressively controlled the Adriatic Sea,

how it became the most powerful medieval empire

of its time,

controlling most of the sea routes

from the east to the south.

But you can even do other things,

because in these maritime routes,

there are regular patterns.

You can go one step beyond

and actually create a simulation system,

create a Mediterranean simulator

which is capable actually of reconstructing

even the information we are missing,

which would enable us to have
questions you could ask

like if you were using a route planner.

"If I am in Corfu in June 1323

and want to go to Constantinople,

where can I take a boat?"

Probably we can answer this question

with one or two or three days' precision.

"How much will it cost?"

"What are the chance of encountering pirates?"

Of course, you understand,

the central scientific challenge
of a project like this one

is qualifying, quantifying and representing

uncertainty and inconsistency
at each step of this process.

There are errors everywhere,

errors in the document, it's
the wrong name of the captain,

some of the boats never actually took to sea.

There are errors in translation, interpretative biases,

and on top of that, if you add algorithmic processes,

you're going to have errors in recognition,

errors in extraction,

so you have very, very uncertain data.

So how can we detect and
correct these inconsistencies?

How can we represent that form of uncertainty?

It's difficult. One thing you can do

is document each step of the process,

not only coding the historical information

but what we call the meta-historical information,

how is historical knowledge constructed,

documenting each step.

That will not guarantee that we actually converge

toward a single story of Venice,

but probably we can actually reconstruct

a fully documented potential story of Venice.

Maybe there's not a single map.

Maybe there are several maps.

The system should allow for that,

because we have to deal with
a new form of uncertainty,

which is really new for this type of giant databases.

And how should we communicate

this new research to a large audience?

Again, Venice is extraordinary for that.

With the millions of visitors that come every year,

it's actually one of the best places

to try to invent the museum of the future.

Imagine, horizontally you see the reconstructed map

of a given year,

and vertically, you see the document

that served the reconstruction,

paintings, for instance.

Imagine an immersive system that permits

to go and dive and reconstruct
the Venice of a given year,

some experience you could share within a group.

On the contrary, imagine actually that you start

from a document, a Venetian manuscript,

and you show, actually, what
you can construct out of it,

how it is decoded,

how the context of that document can be recreated.

This is an image from an exhibit

which is currently conducted in Geneva

with that type of system.

So to conclude, we can say that

research in the humanities is about to undergo

an evolution which is maybe similar

to what happened to life sciences 30 years ago.

It's really a question of scale.

We see projects which are

much beyond any single research team can do,

and this is really new for the humanities,

which very often take the habit of working

in small groups or only with a couple of researchers.

When you visit the Archivio di Stato,

you feel this is beyond what any single team can do,

and that should be a joint and common effort.

So what we must do for this paradigm shift

is actually foster a new generation

of "digital humanists"

that are going to be ready for this shift.

I thank you very much.

(Applause)