WEBVTT

00:00:00.000 --> 00:00:03.000
Throughout the history of computers

00:00:03.000 --> 00:00:06.000
we've been striving to shorten the gap between us

00:00:06.000 --> 00:00:08.000
and digital information,

00:00:08.000 --> 00:00:10.000
the gap between our physical world

00:00:10.000 --> 00:00:12.000
and the world in the screen

00:00:12.000 --> 00:00:14.000
where our imagination can go wild.

00:00:14.000 --> 00:00:18.000
And this gap has become shorter,

00:00:18.000 --> 00:00:21.000
shorter, and even shorter,

00:00:21.000 --> 00:00:23.000
and now this gap is shortened down

00:00:23.000 --> 00:00:24.000
to less than a millimeter,

00:00:24.000 --> 00:00:26.000
the thickness of a touch-screen glass,

00:00:26.000 --> 00:00:27.000
and the power of computing

00:00:27.000 --> 00:00:32.000
has become accessible to everyone.

NOTE Paragraph

00:00:32.000 --> 00:00:37.000
But I wondered, what if there could be no boundary at all?

00:00:37.000 --> 00:00:41.000
I started to imagine what this would look like.

00:00:41.000 --> 00:00:44.000
First, I created this tool

00:00:44.000 --> 00:00:46.000
which penetrates into the digital space,

00:00:46.000 --> 00:00:48.000
so when you press it hard on the screen,

00:00:48.000 --> 00:00:51.000
it transfers its physical body into pixels.

00:00:51.000 --> 00:00:54.000
Designers can materialize their ideas

00:00:54.000 --> 00:00:55.000
directly in 3D,

00:00:55.000 --> 00:00:58.000
and surgeons can practice on virtual organs

00:00:58.000 --> 00:01:00.000
underneath the screen.

00:01:00.000 --> 00:01:05.000
So with this tool, this boundary has been broken.

NOTE Paragraph

00:01:05.000 --> 00:01:08.000
But our two hands still remain outside the screen.

00:01:08.000 --> 00:01:10.000
How can you reach inside and interact

00:01:10.000 --> 00:01:12.000
with the digital information

00:01:12.000 --> 00:01:15.000
using the full dexterity of our hands?

00:01:15.000 --> 00:01:17.000
At Microsoft Applied Sciences,

00:01:17.000 --> 00:01:19.000
along with my mentor Cati Boulanger,

00:01:19.000 --> 00:01:20.000
I redesigned the computer

00:01:20.000 --> 00:01:23.000
and turned a little space above the keyboard

00:01:23.000 --> 00:01:26.000
into a digital workspace.

00:01:26.000 --> 00:01:28.000
By combining a transparent display and depth cameras

00:01:28.000 --> 00:01:30.000
for sensing your fingers and face,

00:01:30.000 --> 00:01:32.000
now you can lift up your hands from the keyboard

00:01:32.000 --> 00:01:35.000
and reach inside this 3D space

00:01:35.000 --> 00:01:37.000
and grab pixels with your bare hands.

NOTE Paragraph

00:01:37.000 --> 00:01:40.000
(Applause)

NOTE Paragraph

00:01:40.000 --> 00:01:45.000
Because windows and files have a position in the real space,

00:01:45.000 --> 00:01:48.000
selecting them is as easy as grabbing a book off your shelf.

00:01:48.000 --> 00:01:50.000
Then you can flip through this book

00:01:50.000 --> 00:01:52.000
while highlighting the lines, words

00:01:52.000 --> 00:01:54.000
on the virtual touch pad below each floating window.

00:01:54.000 --> 00:01:57.000
Architects can stretch or rotate the models

00:01:57.000 --> 00:01:59.000
with their two hands directly.

00:01:59.000 --> 00:02:01.000
So in these examples,

00:02:01.000 --> 00:02:04.000
we are reaching into the digital world.

NOTE Paragraph

00:02:04.000 --> 00:02:06.000
But how about reversing its role

00:02:06.000 --> 00:02:11.000
and having the digital information reach us instead?

00:02:11.000 --> 00:02:13.000
I'm sure many of us have the experience

00:02:13.000 --> 00:02:15.000
of buying and returning items online.

00:02:15.000 --> 00:02:17.000
But now you don't have to worry about it.

00:02:17.000 --> 00:02:22.000
What I got here is an online augmented fitting room.

00:02:22.000 --> 00:02:23.000
This is a view that you get from

00:02:23.000 --> 00:02:25.000
head-mounted or see-through display

00:02:25.000 --> 00:02:30.000
when the system understands the geometry of your body.

NOTE Paragraph

00:02:30.000 --> 00:02:34.000
Taking this idea further, I started to think,

00:02:34.000 --> 00:02:37.000
instead of just seeing these pixels in our space,

00:02:37.000 --> 00:02:39.000
how can we make it physical

00:02:39.000 --> 00:02:43.000
so that we can touch and feel it?

00:02:43.000 --> 00:02:46.000
What would such a future look like?

00:02:46.000 --> 00:02:50.000
At MIT Media Lab, along with my advisor Hiroshi Ishii

00:02:50.000 --> 00:02:52.000
and my collaborator Rehmi Post,

00:02:52.000 --> 00:02:55.000
we created this one physical pixel.

00:02:55.000 --> 00:02:58.000
Well, in this case, this spherical magnet

00:02:58.000 --> 00:03:00.000
acts like a 3D pixel in our space,

00:03:00.000 --> 00:03:03.000
which means that both computers and people

00:03:03.000 --> 00:03:05.000
can move this object to anywhere

00:03:05.000 --> 00:03:07.000
within this little 3D space.

00:03:07.000 --> 00:03:11.000
What we did was essentially canceling gravity

00:03:11.000 --> 00:03:13.000
and controlling the movement by combining

00:03:13.000 --> 00:03:16.000
magnetic levitation and mechanical actuation

00:03:16.000 --> 00:03:18.000
and sensing technologies.

00:03:18.000 --> 00:03:21.000
And by digitally programming the object,

00:03:21.000 --> 00:03:23.000
we are liberating the objecting from constraints

00:03:23.000 --> 00:03:27.000
of time and space, which means that now,

00:03:27.000 --> 00:03:31.000
human motions can be recorded and played back

00:03:31.000 --> 00:03:34.000
and left permanently in the physical world.

00:03:34.000 --> 00:03:37.000
So choreography can be taught physically over distance

00:03:37.000 --> 00:03:40.000
and Michael Jordan's famous shooting can be replicated

00:03:40.000 --> 00:03:43.000
over and over as a physical reality.

00:03:43.000 --> 00:03:46.000
Students can use this as a tool

00:03:46.000 --> 00:03:48.000
to learn about the complex concepts

00:03:48.000 --> 00:03:52.000
such as the planetary motion, physics,

00:03:52.000 --> 00:03:54.000
and unlike computer screens or textbooks,

00:03:54.000 --> 00:03:57.000
this is a real, tangible experience

00:03:57.000 --> 00:04:01.000
that you can touch and feel, and it's very powerful.

00:04:01.000 --> 00:04:03.000
And what's more exciting

00:04:03.000 --> 00:04:06.000
than just turning what's currently in the computer physical

00:04:06.000 --> 00:04:09.000
is to start imagining how programming the world

00:04:09.000 --> 00:04:13.000
will alter even our daily physical activities.

NOTE Paragraph

00:04:15.000 --> 00:04:17.000
(Laughter)

NOTE Paragraph

00:04:17.000 --> 00:04:20.000
As you can see, the digital information

00:04:20.000 --> 00:04:22.000
will not just show us something

00:04:22.000 --> 00:04:24.000
but it will start directly acting upon us

00:04:24.000 --> 00:04:26.000
as a part of our physical surroundings

00:04:26.000 --> 00:04:30.000
without disconnecting ourselves from our world.

NOTE Paragraph

00:04:30.000 --> 00:04:35.000
Today, we started by talking about the boundary,

00:04:35.000 --> 00:04:37.000
but if we remove this boundary,

00:04:37.000 --> 00:04:41.000
the only boundary left is our imagination.

NOTE Paragraph

00:04:41.000 --> 00:04:42.000
Thank you.

NOTE Paragraph

00:04:42.000 --> 00:04:48.000
(Applause)