1
00:00:01,280 --> 00:00:04,936
In my lab, we build
autonomous aerial robots

2
00:00:04,960 --> 00:00:06,840
like the one you see flying here.

3
00:00:08,720 --> 00:00:12,416
Unlike the commercially available drones
that you can buy today,

4
00:00:12,440 --> 00:00:15,080
this robot doesn't have any GPS on board.

5
00:00:16,160 --> 00:00:17,376
So without GPS,

6
00:00:17,400 --> 00:00:20,680
it's hard for robots like this
to determine their position.

7
00:00:22,240 --> 00:00:26,976
This robot uses onboard sensors,
cameras and laser scanners,

8
00:00:27,000 --> 00:00:28,696
to scan the environment.

9
00:00:28,720 --> 00:00:31,776
It detects features from the environment,

10
00:00:31,800 --> 00:00:34,536
and it determines where it is
relative to those features,

11
00:00:34,560 --> 00:00:36,696
using a method of triangulation.

12
00:00:36,720 --> 00:00:40,176
And then it can assemble
all these features into a map,

13
00:00:40,200 --> 00:00:41,936
like you see behind me.

14
00:00:41,960 --> 00:00:45,896
And this map then allows the robot
to understand where the obstacles are

15
00:00:45,920 --> 00:00:48,640
and navigate in a collision-free manner.

16
00:00:49,160 --> 00:00:51,256
What I want to show you next

17
00:00:51,280 --> 00:00:54,496
is a set of experiments
we did inside our laboratory,

18
00:00:54,520 --> 00:00:58,000
where this robot was able
to go for longer distances.

19
00:00:58,400 --> 00:01:03,416
So here you'll see, on the top right,
what the robot sees with the camera.

20
00:01:03,440 --> 00:01:04,656
And on the main screen --

21
00:01:04,680 --> 00:01:07,136
and of course this is sped up
by a factor of four --

22
00:01:07,160 --> 00:01:09,827
on the main screen you'll see
the map that it's building.

23
00:01:09,851 --> 00:01:14,136
So this is a high-resolution map
of the corridor around our laboratory.

24
00:01:14,160 --> 00:01:16,496
And in a minute
you'll see it enter our lab,

25
00:01:16,520 --> 00:01:19,376
which is recognizable
by the clutter that you see.

26
00:01:19,400 --> 00:01:20,416
(Laughter)

27
00:01:20,440 --> 00:01:22,447
But the main point I want to convey to you

28
00:01:22,472 --> 00:01:26,056
is that these robots are capable
of building high-resolution maps

29
00:01:26,080 --> 00:01:28,576
at five centimeters resolution,

30
00:01:28,600 --> 00:01:32,776
allowing somebody who is outside the lab,
or outside the building

31
00:01:32,800 --> 00:01:36,016
to deploy these
without actually going inside,

32
00:01:36,040 --> 00:01:39,800
and trying to infer
what happens inside the building.

33
00:01:40,400 --> 00:01:42,640
Now there's one problem
with robots like this.

34
00:01:43,600 --> 00:01:45,800
The first problem is it's pretty big.

35
00:01:46,120 --> 00:01:47,800
Because it's big, it's heavy.

36
00:01:48,640 --> 00:01:51,680
And these robots consume
about 100 watts per pound.

37
00:01:52,360 --> 00:01:54,640
And this makes for
a very short mission life.

38
00:01:56,000 --> 00:01:57,456
The second problem

39
00:01:57,480 --> 00:02:01,376
is that these robots have onboard sensors
that end up being very expensive --

40
00:02:01,400 --> 00:02:04,840
a laser scanner, a camera
and the processors.

41
00:02:05,280 --> 00:02:08,320
That drives up the cost of this robot.

42
00:02:09,440 --> 00:02:12,096
So we asked ourselves a question:

43
00:02:12,120 --> 00:02:15,896
what consumer product
can you buy in an electronics store

44
00:02:15,920 --> 00:02:22,200
that is inexpensive, that's lightweight,
that has sensing onboard and computation?

45
00:02:24,080 --> 00:02:26,736
And we invented the flying phone.

46
00:02:26,760 --> 00:02:28,696
(Laughter)

47
00:02:28,720 --> 00:02:34,896
So this robot uses a Samsung Galaxy
smartphone that you can buy off the shelf,

48
00:02:34,920 --> 00:02:38,936
and all you need is an app that you
can download from our app store.

49
00:02:38,960 --> 00:02:43,176
And you can see this robot
reading the letters, "TED" in this case,

50
00:02:43,200 --> 00:02:46,136
looking at the corners
of the "T" and the "E"

51
00:02:46,160 --> 00:02:49,640
and then triangulating off of that,
flying autonomously.

52
00:02:50,720 --> 00:02:53,976
That joystick is just there
to make sure if the robot goes crazy,

53
00:02:54,000 --> 00:02:55,416
Giuseppe can kill it.

54
00:02:55,440 --> 00:02:57,080
(Laughter)

55
00:02:58,920 --> 00:03:02,736
In addition to building
these small robots,

56
00:03:02,760 --> 00:03:07,560
we also experiment with aggressive
behaviors, like you see here.

57
00:03:07,920 --> 00:03:13,216
So this robot is now traveling
at two to three meters per second,

58
00:03:13,240 --> 00:03:16,736
pitching and rolling aggressively
as it changes direction.

59
00:03:16,760 --> 00:03:21,016
The main point is we can have
smaller robots that can go faster

60
00:03:21,040 --> 00:03:24,000
and then travel in these
very unstructured environments.

61
00:03:25,120 --> 00:03:27,176
And in this next video,

62
00:03:27,200 --> 00:03:33,096
just like you see this bird, an eagle,
gracefully coordinating its wings,

63
00:03:33,120 --> 00:03:37,416
its eyes and feet
to grab prey out of the water,

64
00:03:37,440 --> 00:03:39,336
our robot can go fishing, too.

65
00:03:39,360 --> 00:03:40,856
(Laughter)

66
00:03:40,880 --> 00:03:44,936
In this case, this is a Philly cheesesteak
hoagie that it's grabbing out of thin air.

67
00:03:44,960 --> 00:03:47,360
(Laughter)

68
00:03:47,680 --> 00:03:50,976
So you can see this robot
going at about three meters per second,

69
00:03:51,000 --> 00:03:56,136
which is faster than walking speed,
coordinating its arms, its claws

70
00:03:56,160 --> 00:04:00,280
and its flight with split-second timing
to achieve this maneuver.

71
00:04:02,120 --> 00:04:03,336
In another experiment,

72
00:04:03,360 --> 00:04:07,016
I want to show you
how the robot adapts its flight

73
00:04:07,040 --> 00:04:09,416
to control its suspended payload,

74
00:04:09,440 --> 00:04:13,240
whose length is actually larger
than the width of the window.

75
00:04:13,680 --> 00:04:15,376
So in order to accomplish this,

76
00:04:15,400 --> 00:04:19,096
it actually has to pitch
and adjust the altitude

77
00:04:19,120 --> 00:04:21,440
and swing the payload through.

78
00:04:26,920 --> 00:04:29,216
But of course we want
to make these even smaller,

79
00:04:29,240 --> 00:04:32,256
and we're inspired
in particular by honeybees.

80
00:04:32,280 --> 00:04:35,536
So if you look at honeybees,
and this is a slowed down video,

81
00:04:35,560 --> 00:04:39,280
they're so small,
the inertia is so lightweight --

82
00:04:39,960 --> 00:04:41,136
(Laughter)

83
00:04:41,160 --> 00:04:44,696
that they don't care --
they bounce off my hand, for example.

84
00:04:44,720 --> 00:04:47,880
This is a little robot
that mimics the honeybee behavior.

85
00:04:48,600 --> 00:04:49,816
And smaller is better,

86
00:04:49,840 --> 00:04:53,376
because along with the small size
you get lower inertia.

87
00:04:53,400 --> 00:04:54,936
Along with lower inertia --

88
00:04:54,960 --> 00:04:57,816
(Robot buzzing, laughter)

89
00:04:57,840 --> 00:05:00,656
along with lower inertia,
you're resistant to collisions.

90
00:05:00,680 --> 00:05:02,400
And that makes you more robust.

91
00:05:03,800 --> 00:05:06,456
So just like these honeybees,
we build small robots.

92
00:05:06,480 --> 00:05:09,856
And this particular one
is only 25 grams in weight.

93
00:05:09,880 --> 00:05:12,040
It consumes only six watts of power.

94
00:05:12,440 --> 00:05:14,976
And it can travel
up to six meters per second.

95
00:05:15,000 --> 00:05:17,336
So if I normalize that to its size,

96
00:05:17,360 --> 00:05:21,000
it's like a Boeing 787 traveling
ten times the speed of sound.

97
00:05:24,000 --> 00:05:26,096
(Laughter)

98
00:05:26,120 --> 00:05:28,040
And I want to show you an example.

99
00:05:28,840 --> 00:05:34,096
This is probably the first planned mid-air
collision, at one-twentieth normal speed.

100
00:05:34,120 --> 00:05:36,978
These are going at a relative speed
of two meters per second,

101
00:05:37,002 --> 00:05:39,482
and this illustrates the basic principle.

102
00:05:40,200 --> 00:05:45,176
The two-gram carbon fiber cage around it
prevents the propellers from entangling,

103
00:05:45,200 --> 00:05:50,496
but essentially the collision is absorbed
and the robot responds to the collisions.

104
00:05:50,520 --> 00:05:53,080
And so small also means safe.

105
00:05:53,400 --> 00:05:55,416
In my lab, as we developed these robots,

106
00:05:55,440 --> 00:05:57,060
we start off with these big robots

107
00:05:57,084 --> 00:05:59,896
and then now we're down
to these small robots.

108
00:05:59,920 --> 00:06:03,376
And if you plot a histogram
of the number of Band-Aids we've ordered

109
00:06:03,400 --> 00:06:05,976
in the past, that sort of tailed off now.

110
00:06:06,000 --> 00:06:07,960
Because these robots are really safe.

111
00:06:08,760 --> 00:06:11,216
The small size has some disadvantages,

112
00:06:11,240 --> 00:06:15,320
and nature has found a number of ways
to compensate for these disadvantages.

113
00:06:15,960 --> 00:06:19,960
The basic idea is they aggregate
to form large groups, or swarms.

114
00:06:20,320 --> 00:06:24,296
So, similarly, in our lab,
we try to create artificial robot swarms.

115
00:06:24,320 --> 00:06:25,701
And this is quite challenging

116
00:06:25,725 --> 00:06:29,045
because now you have to think
about networks of robots.

117
00:06:29,360 --> 00:06:30,656
And within each robot,

118
00:06:30,680 --> 00:06:36,296
you have to think about the interplay
of sensing, communication, computation --

119
00:06:36,320 --> 00:06:41,280
and this network then becomes
quite difficult to control and manage.

120
00:06:42,160 --> 00:06:45,456
So from nature we take away
three organizing principles

121
00:06:45,480 --> 00:06:48,640
that essentially allow us
to develop our algorithms.

122
00:06:49,640 --> 00:06:54,176
The first idea is that robots
need to be aware of their neighbors.

123
00:06:54,200 --> 00:06:57,640
They need to be able to sense
and communicate with their neighbors.

124
00:06:58,040 --> 00:07:00,696
So this video illustrates the basic idea.

125
00:07:00,720 --> 00:07:02,016
You have four robots --

126
00:07:02,040 --> 00:07:06,280
one of the robots has actually been
hijacked by a human operator, literally.

127
00:07:07,217 --> 00:07:09,456
But because the robots
interact with each other,

128
00:07:09,480 --> 00:07:11,136
they sense their neighbors,

129
00:07:11,160 --> 00:07:12,456
they essentially follow.

130
00:07:12,480 --> 00:07:17,840
And here there's a single person
able to lead this network of followers.

131
00:07:20,000 --> 00:07:25,056
So again, it's not because all the robots
know where they're supposed to go.

132
00:07:25,080 --> 00:07:29,400
It's because they're just reacting
to the positions of their neighbors.

133
00:07:31,720 --> 00:07:35,840
(Laughter)

134
00:07:36,280 --> 00:07:41,520
So the next experiment illustrates
the second organizing principle.

135
00:07:42,920 --> 00:07:46,720
And this principle has to do
with the principle of anonymity.

136
00:07:47,400 --> 00:07:51,696
Here the key idea is that

137
00:07:51,720 --> 00:07:55,960
the robots are agnostic
to the identities of their neighbors.

138
00:07:56,440 --> 00:07:59,056
They're asked to form a circular shape,

139
00:07:59,080 --> 00:08:02,376
and no matter how many robots
you introduce into the formation,

140
00:08:02,400 --> 00:08:04,976
or how many robots you pull out,

141
00:08:05,000 --> 00:08:08,136
each robot is simply
reacting to its neighbor.

142
00:08:08,160 --> 00:08:13,136
It's aware of the fact that it needs
to form the circular shape,

143
00:08:13,160 --> 00:08:14,936
but collaborating with its neighbors

144
00:08:14,960 --> 00:08:18,680
it forms the shape
without central coordination.

145
00:08:19,520 --> 00:08:21,936
Now if you put these ideas together,

146
00:08:21,960 --> 00:08:25,856
the third idea is that we
essentially give these robots

147
00:08:25,880 --> 00:08:30,176
mathematical descriptions
of the shape they need to execute.

148
00:08:30,200 --> 00:08:33,696
And these shapes can be varying
as a function of time,

149
00:08:33,720 --> 00:08:38,216
and you'll see these robots
start from a circular formation,

150
00:08:38,240 --> 00:08:41,496
change into a rectangular formation,
stretch into a straight line,

151
00:08:41,520 --> 00:08:42,895
back into an ellipse.

152
00:08:42,919 --> 00:08:46,536
And they do this with the same
kind of split-second coordination

153
00:08:46,560 --> 00:08:49,840
that you see in natural swarms, in nature.

154
00:08:51,080 --> 00:08:53,216
So why work with swarms?

155
00:08:53,240 --> 00:08:57,360
Let me tell you about two applications
that we are very interested in.

156
00:08:58,160 --> 00:09:00,536
The first one has to do with agriculture,

157
00:09:00,560 --> 00:09:03,920
which is probably the biggest problem
that we're facing worldwide.

158
00:09:04,760 --> 00:09:06,016
As you well know,

159
00:09:06,040 --> 00:09:09,560
one in every seven persons
in this earth is malnourished.

160
00:09:09,920 --> 00:09:13,400
Most of the land that we can cultivate
has already been cultivated.

161
00:09:13,960 --> 00:09:17,176
And the efficiency of most systems
in the world is improving,

162
00:09:17,200 --> 00:09:20,720
but our production system
efficiency is actually declining.

163
00:09:21,080 --> 00:09:25,296
And that's mostly because of water
shortage, crop diseases, climate change

164
00:09:25,320 --> 00:09:26,840
and a couple of other things.

165
00:09:27,360 --> 00:09:28,840
So what can robots do?

166
00:09:29,200 --> 00:09:33,816
Well, we adopt an approach that's
called Precision Farming in the community.

167
00:09:33,840 --> 00:09:39,216
And the basic idea is that we fly
aerial robots through orchards,

168
00:09:39,240 --> 00:09:42,360
and then we build
precision models of individual plants.

169
00:09:42,829 --> 00:09:44,496
So just like personalized medicine,

170
00:09:44,520 --> 00:09:49,336
while you might imagine wanting
to treat every patient individually,

171
00:09:49,360 --> 00:09:53,056
what we'd like to do is build
models of individual plants

172
00:09:53,080 --> 00:09:57,216
and then tell the farmer
what kind of inputs every plant needs --

173
00:09:57,240 --> 00:10:01,680
the inputs in this case being water,
fertilizer and pesticide.

174
00:10:02,640 --> 00:10:06,256
Here you'll see robots
traveling through an apple orchard,

175
00:10:06,280 --> 00:10:08,536
and in a minute you'll see
two of its companions

176
00:10:08,560 --> 00:10:10,370
doing the same thing on the left side.

177
00:10:10,800 --> 00:10:14,456
And what they're doing is essentially
building a map of the orchard.

178
00:10:14,480 --> 00:10:17,296
Within the map is a map
of every plant in this orchard.

179
00:10:17,320 --> 00:10:18,976
(Robot buzzing)

180
00:10:19,000 --> 00:10:20,896
Let's see what those maps look like.

181
00:10:20,920 --> 00:10:25,216
In the next video, you'll see the cameras
that are being used on this robot.

182
00:10:25,240 --> 00:10:28,480
On the top-left is essentially
a standard color camera.

183
00:10:29,640 --> 00:10:32,936
On the left-center is an infrared camera.

184
00:10:32,960 --> 00:10:36,736
And on the bottom-left
is a thermal camera.

185
00:10:36,760 --> 00:10:40,096
And on the main panel, you're seeing
a three-dimensional reconstruction

186
00:10:40,120 --> 00:10:46,240
of every tree in the orchard
as the sensors fly right past the trees.

187
00:10:47,640 --> 00:10:51,680
Armed with information like this,
we can do several things.

188
00:10:52,200 --> 00:10:56,456
The first and possibly the most important
thing we can do is very simple:

189
00:10:56,480 --> 00:10:58,920
count the number of fruits on every tree.

190
00:10:59,520 --> 00:11:04,056
By doing this, you tell the farmer
how many fruits she has in every tree

191
00:11:04,080 --> 00:11:08,336
and allow her to estimate
the yield in the orchard,

192
00:11:08,360 --> 00:11:11,200
optimizing the production
chain downstream.

193
00:11:11,640 --> 00:11:13,256
The second thing we can do

194
00:11:13,280 --> 00:11:17,776
is take models of plants, construct
three-dimensional reconstructions,

195
00:11:17,800 --> 00:11:20,336
and from that estimate the canopy size,

196
00:11:20,360 --> 00:11:24,136
and then correlate the canopy size
to the amount of leaf area on every plant.

197
00:11:24,160 --> 00:11:26,336
And this is called the leaf area index.

198
00:11:26,360 --> 00:11:28,296
So if you know this leaf area index,

199
00:11:28,320 --> 00:11:33,776
you essentially have a measure of how much
photosynthesis is possible in every plant,

200
00:11:33,800 --> 00:11:36,680
which again tells you
how healthy each plant is.

201
00:11:37,520 --> 00:11:41,736
By combining visual
and infrared information,

202
00:11:41,760 --> 00:11:45,056
we can also compute indices such as NDVI.

203
00:11:45,080 --> 00:11:47,896
And in this particular case,
you can essentially see

204
00:11:47,920 --> 00:11:50,936
there are some crops that are
not doing as well as other crops.

205
00:11:50,960 --> 00:11:55,016
This is easily discernible from imagery,

206
00:11:55,040 --> 00:11:57,256
not just visual imagery but combining

207
00:11:57,280 --> 00:12:00,056
both visual imagery and infrared imagery.

208
00:12:00,080 --> 00:12:01,416
And then lastly,

209
00:12:01,440 --> 00:12:05,456
one thing we're interested in doing is
detecting the early onset of chlorosis --

210
00:12:05,480 --> 00:12:06,976
and this is an orange tree --

211
00:12:07,000 --> 00:12:09,560
which is essentially seen
by yellowing of leaves.

212
00:12:09,880 --> 00:12:13,776
But robots flying overhead
can easily spot this autonomously

213
00:12:13,800 --> 00:12:16,736
and then report to the farmer
that he or she has a problem

214
00:12:16,760 --> 00:12:18,280
in this section of the orchard.

215
00:12:18,800 --> 00:12:21,496
Systems like this can really help,

216
00:12:21,520 --> 00:12:27,336
and we're projecting yields
that can improve by about ten percent

217
00:12:27,360 --> 00:12:30,576
and, more importantly, decrease
the amount of inputs such as water

218
00:12:30,600 --> 00:12:33,880
by 25 percent by using
aerial robot swarms.

219
00:12:35,200 --> 00:12:40,936
Lastly, I want you to applaud
the people who actually create the future,

220
00:12:40,960 --> 00:12:45,880
Yash Mulgaonkar, Sikang Liu
and Giuseppe Loianno,

221
00:12:45,920 --> 00:12:49,416
who are responsible for the three
demonstrations that you saw.

222
00:12:49,440 --> 00:12:50,616
Thank you.

223
00:12:50,640 --> 00:12:56,560
(Applause)