Brain-to-brain communication has arrived. How we did it

You may remember neuroscientist Miguel Nicolelis — he built the brain-controlled exoskeleton that allowed a paralyzed man to kick the first ball of the 2014 World Cup. What’s he working on now? Building ways for two minds (rats and monkeys, for now) to send messages brain to brain. Watch to the end for an experiment that, as he says, will go to "the limit of your imagination."

On June 12, 2014, precisely at 3:33

in a balmy winter afternoon
in São Paulo, Brazil,

a typical South American winter afternoon,

this kid, this young man
that you see celebrating here

like he had scored a goal,

Juliano Pinto, 29 years old,
accomplished a magnificent deed.

Despite being paralyzed

and not having any sensation
from mid-chest to the tip of his toes

as the result of a car crash six years ago
that killed his brother

and produced a complete spinal cord lesion
that left Juliano in a wheelchair,

Juliano rose to the occasion,
and on this day did something

that pretty much everybody that saw him
in the six years deemed impossible.

Juliano Pinto delivered the opening kick

of the 2014 Brazilian
World Soccer Cup here

just by thinking.

He could not move his body,

but he could imagine the movements
needed to kick a ball.

He was an athlete before the lesion.
He's a para-athlete right now.

He's going to be in the Paralympic Games,
I hope, in a couple years.

But what the spinal cord lesion
did not rob from Juliano

was his ability to dream.

And dream he did that afternoon,
for a stadium of about 75,000 people

and an audience of close to a billion
watching on TV.

And that kick crowned, basically,
30 years of basic research

studying how the brain,

how this amazing universe
that we have between our ears

that is only comparable to universe
that we have above our head

because it has about 100 billion elements

talking to each other
through electrical brainstorms,

what Juliano accomplished
took 30 years to imagine in laboratories

and about 15 years to plan.

When John Chapin and I,
15 years ago, proposed in a paper

that we would build something
that we called a brain-machine interface,

meaning connecting a brain to devices

so that animals and humans
could just move these devices,

no matter how far they are
from their own bodies,

just by imagining what they want to do,

our colleagues told us that
we actually needed professional help,

of the psychiatry variety.

And despite that,
a Scot and a Brazilian persevered,

because that's how we were raised
in our respective countries,

and for 12, 15 years,

we made demonstration after demonstration
suggesting that this was possible.

And a brain-machine interface
is not rocket science,

it's just brain research.

It's nothing but using sensors

to read the electrical brainstorms
that a brain is producing

to generate the motor commands

that have to be downloaded
to the spinal cord,

so we projected sensors that can read

hundreds and now thousands
of these brain cells simultaneously,

and extract from these electrical signals

the motor planning
that the brain is generating

to actually make us move into space.

And by doing that, we converted
these signals into digital commands

that any mechanical, electronic,
or even a virtual device can understand

so that the subject can imagine
what he, she or it wants to make move,

and the device obeys that brain command.

By sensorizing these devices
with lots of different types of sensors,

as you are going to see in a moment,

we actually sent messages
back to the brain to confirm

that that voluntary motor will
was being enacted, no matter where --

next to the subject, next door,
or across the planet.

And as this message gave feedback
back to the brain,

the brain realized its goal:
to make us move.

So this is just one experiment
that we published a few years ago,

where a monkey, without moving its body,

learned to control the movements
of an avatar arm,

a virtual arm that doesn't exist.

What you're listening to
is the sound of the brain of this monkey

as it explores three different
visually identical spheres

in virtual space.

And to get a reward,
a drop of orange juice that monkeys love,

this animal has to detect,
select one of these objects

by touching,

not by seeing it, by touching it,

because every time this virtual hand
touches one of the objects,

an electrical pulse goes back
to the brain of the animal

describing the fine texture
of the surface of this object,

so the animal can judge what is
the correct object that he has to grab,

and if he does that, he gets a reward
without moving a muscle.

The perfect Brazilian lunch:

not moving a muscle
and getting your orange juice.

So as we saw this happening,

we actually came and proposed the idea
that we had published 15 years ago.

We reenacted this paper.

We got it out of the drawers,

and we proposed that perhaps we could get
a human being that is paralyzed

to actually use the brain-machine
interface to regain mobility.

The idea was that if you suffered --

and that can happen to any one of us.

Let me tell you, it's very sudden.

It's a millisecond of a collision,

a car accident that
transforms your life completely.

If you have a complete lesion
of the spinal cord,

you cannot move because your brainstorms
cannot reach your muscles.

However, your brainstorms
continue to be generated in your head.

Paraplegic, quadriplegic patients
dream about moving every night.

They have that inside their head.

The problem is how
to get that code out of it

and make the movement be created again.

So what we proposed was,
let's create a new body.

Let's create a robotic vest.

And that's exactly why Juliano could
kick that ball just by thinking,

because he was wearing
the first brain-controlled robotic vest

that can be used by paraplegic,
quadriplegic patients to move

and to regain feedback.

That was the original idea, 15 years ago.

What I'm going to show you is how
156 people from 25 countries

all over the five continents
of this beautiful Earth,

dropped their lives,
dropped their patents,

dropped their dogs, wives,
kids, school, jobs,

and congregated to come to Brazil
for 18 months to actually get this done.

Because a couple years after Brazil
was awarded the World Cup,

we heard that the Brazilian government
wanted to do something meaningful

in the opening ceremony

in the country that reinvented
and perfected soccer

until we met the Germans, of course.

(Laughter)

But that's a different talk,

and a different neuroscientist
needs to talk about that.

But what Brazil wanted to do
is to showcase

a completely different country,

a country that values science
and technology,

and can give a gift to millions,
25 million people around the world

that cannot move any longer
because of a spinal cord injury.

Well, we went to the Brazilian government
and to FIFA and proposed,

well, let's have the kickoff
of the 2014 World Cup

be given by a Brazilian paraplegic

using a brain-controlled exoskeleton
that allows him to kick the ball

and to feel the contact of the ball.

They looked at us,
thought that we were completely nuts,

and said, "Okay, let's try."

We had 18 months to do everything
from zero, from scratch.

We had no exoskeleton, we had no patients,

we had nothing done.

These people came all together

and in 18 months, we got eight patients
in a routine of training

and basically built from nothing this guy,

that we call Bra-Santos Dumont 1.

The first brain-controlled
exoskeleton to be built

was named after the most famous
Brazilian scientist ever,

Alberto Santos Dumont,

who, on October 19, 1901,
created and flew himself

the first controlled airship on air
in Paris for a million people to see.

Sorry, my American friends,

I live in North Carolina,

but it was two years
before the Wright Brothers flew

on the coast of North Carolina.

(Applause)

Flight control is Brazilian.
(Laughter)

So we went together with these guys

and we basically put
this exoskeleton together,

15 degrees of freedom,
hydraulic machine

that can be commanded by brain signals

recorded by a non-invasive technology
called electroencephalography

that can basically allow the patient
to imagine the movements

and send his commands
to the controls, the motors,

and get it done.

This exoskeleton was covered
with an artificial skin

invented by Gordon Cheng,
one of my greatest friends, in Munich,

to allow sensation from the joints moving
and the foot touching the ground

to be delivered back to the patient
through a vest, a shirt.

It is a smart shirt
with micro-vibrating elements

that basically delivers the feedback
and fools the patient's brain

by creating a sensation that it is not
a machine that is carrying him,

but it is he who is walking again.

So we got this going,
and what you'll see here

is the first time one of our patients,
Bruno, actually walked.

And he takes a few seconds
because we are setting everything,

and you are going to see a blue light
cutting in front of the helmet

because Bruno is going to imagine
the movement that needs to be performed,

the computer is going to analyze it,
Bruno is going to certify it,

and when it is certified,

the device starts moving
under the command of Bruno's brain.

And he just got it right,
and now he starts walking.

After nine years without
being able to move,

he is walking by himself.

And more than that --

(Applause) --

more than just walking,

he is feeling the ground,

and if the speed of the exo goes up,

he tells us that he is walking again
on the sand of Santos,

the beach resort where he used to go
before he had the accident.

That's why the brain is creating
a new sensation in Bruno's head.

So he walks, and at the end of the walk --
I am running out of time already --

he says, "You know, guys,

I need to borrow this thing from you
when I get married,

because I wanted to walk to the priest

and see my bride and actually
be there by myself.

Of course, he will have it
whenever he wants.

And this is what we wanted to show
during the World Cup, and couldn't,

because for some mysterious reason,
FIFA cut its broadcast in half.

What you are going to see very quickly
is Juliano Pinto in the exo doing the kick

a few minutes before we went to the pitch

and did the real thing
in front of the entire crowd,

and the lights you are going to see
just describe the operation.

Basically, the blue lights pulsating
indicate that the exo is ready to go.

It can receive thoughts
and it can deliver feedback,

and when Juliano
makes the decision to kick the ball,

you are going to see
two streams of green and yellow light

coming from the helmet
and going to the legs,

representing the mental commands
that were taken by the exo

to actually make that happen.

And in basically 13 seconds,

Juliano actually did.

You can see the commands.

He gets ready,
the ball is set, and he kicks.

And the most amazing thing is,

10 seconds after he did that,
and looked at us on the pitch,

he told us, celebrating as you saw,

"I felt the ball."

And that's priceless.

(Applause)

So where is this going to go?

I have two minutes to tell you

that it's going to the limits
of your imagination.

Brain-actuating technology is here.

This is the latest: We just
published this a year ago,

the first brain-to-brain interface

that allows two animals
to exchange mental messages

so that one animal that sees something
coming from the environment

can send a mental SMS,
a torpedo, a neurophysiological torpedo,

to the second animal,

and the second animal performs
the act that he needed to perform

without ever knowing what
the environment was sending as a message,

because the message came
from the first animal's brain.

So this is the first demo.

I'm going to be very quick
because I want to show you the latest.

But what you see here
is the first rat getting informed

by a light that is going to show up
on the left of the cage

that he has to press the left cage
to basically get a reward.

He goes there and does it.

And the same time,
he is sending a mental message

to the second rat
that didn't see any light,

and the second rat,
in 70 percent of the times

is going to press the left lever
and get a reward

without ever experiencing
the light in the retina.

Well, we took this
to a little higher limit

by getting monkeys to collaborate
mentally in a brain net,

basically to donate their brain activity

and combine them to move
the virtual arm that I showed you before,

and what you see here is the first time
the two monkeys combine their brains,

synchronize their brains perfectly
to get this virtual arm to move.

One monkey is controlling the x dimension,

the other monkey
is controlling the y dimension.

But it gets a little more interesting
when you get three monkeys in there

and you ask one monkey to control x and y,

the other monkey to control y and z,

and the third one to control x and z,

and you make them all
play the game together,

moving the arm in 3D into a target
to get the famous Brazilian orange juice.

And they actually do.

The black dot is the average
of all these brains working

in parallel, in real time.

That is the definition
of a biological computer,

interacting by brain activity
and achieving a motor goal.

Where is this going?

We have no idea.

We're just scientists.

(Laughter)

We are paid to be children,

to basically go to the edge
and discover what is out there.

But one thing I know:

One day, in a few decades,

when our grandchildren
surf the Net just by thinking,

or a mother donates her eyesight
to an autistic kid who cannot see,

or somebody speaks because
of a brain-to-brain bypass,

some of you will remember
that it all started on a winter afternoon

in a Brazilian soccer field
with an impossible kick.

Thank you.

(Applause)

Thank you.

Bruno Giussani: Miguel,
thank you for sticking to your time.

I actually would have given you
a couple more minutes,

because there are a couple of points
we want to develop, and, of course,

clearly it seems that we need connected
brains to figure out where this is going.

So let's connect all this together.

So if I'm understanding correctly,

one of the monkeys
is actually getting a signal

and the other monkey
is reacting to that signal

just because the first one is receiving it
and transmitting the neurological impulse.

Miguel Nicolelis:
No, it's a little different.

No monkey knows of the existence
of the other two monkeys.

They are getting a visual feedback in 2D,

but the task they have
to accomplish is 3D.

They have to move an arm
in three dimensions.

But each monkey is only getting
the two dimensions on the video screen

that the monkey controls.

And to get that thing done,

you need at least two monkeys
to synchronize their brains,

but the ideal is three.

So what we found out is that
when one monkey starts slacking down,

the other two monkeys
enhance their performance

to get the guy to come back,

so this adjusts dynamically,

but the global synchrony remains the same.

Now, if you flip
without telling the monkey

the dimensions that each brain
has to control,

like this guy is controlling x and y,

but he should be controlling now y and z,

instantaneously, that animal's brain
forgets about the old dimensions

and it starts concentrating
on the new dimensions.

So what I need to say is
that no Turing machine,

no computer can predict
what a brain net will do.

So we will absorb technology
as part of us.

Technology will never absorb us.

It's simply impossible.

BG: How many times have you tested this?

And how many times
have you succeeded versus failed?

MN: Oh, tens of times.

With the three monkeys?
Oh, several times.

I wouldn't be able to talk about this here
unless I had done it a few times.

And I forgot to mention, because of time,

that just three weeks ago,
a European group

just demonstrated the first
man-to-man brain-to-brain connection.

BG: And how does that play?

MN: There was one bit of information --
big ideas start in a humble way --

but basically the brain activity
of one subject

was transmitted to a second object,
all non-invasive technology.

So the first subject got a message,
like our rats, a visual message,

and transmitted it to the second subject.

The second subject received
a magnetic pulse in the visual cortex,

or a different pulse,
two different pulses.

In one pulse, the subject saw something.

On the other pulse,
he saw something different.

And he was able to verbally indicate

what was the message
the first subject was sending

through the Internet across continents.

Moderator: Wow.
Okay, that's where we are going.

That's the next TED Talk
at the next conference.

Miguel Nicolelis, thank you.
MN: Thank you, Bruno. Thank you.