1
00:00:01,135 --> 00:00:03,111
Everyone is both a learner

2
00:00:03,111 --> 00:00:04,623
and a teacher.

3
00:00:04,623 --> 00:00:07,079
This is me being inspired

4
00:00:07,079 --> 00:00:08,270
by my first tutor,

5
00:00:08,270 --> 00:00:09,215
my mom,

6
00:00:09,215 --> 00:00:11,838
and this is me teaching

7
00:00:11,838 --> 00:00:13,791
Introduction to Artificial Intelligence

8
00:00:13,791 --> 00:00:15,031
to 200 students

9
00:00:15,031 --> 00:00:16,303
at Stanford University.

10
00:00:16,303 --> 00:00:17,759
Now the students and I

11
00:00:17,759 --> 00:00:18,935
enjoyed the class,

12
00:00:18,935 --> 00:00:20,351
but it occurred to me

13
00:00:20,351 --> 00:00:22,062
that while the subject matter

14
00:00:22,062 --> 00:00:23,351
of the class is advanced

15
00:00:23,351 --> 00:00:24,039
and modern,

16
00:00:24,039 --> 00:00:26,358
the teaching technology isn't.

17
00:00:26,358 --> 00:00:28,694
In fact, I use basically

18
00:00:28,694 --> 00:00:31,064
the same technology as

19
00:00:31,064 --> 00:00:33,694
this 14th-century classroom.

20
00:00:33,694 --> 00:00:36,494
Note the textbook,

21
00:00:36,494 --> 00:00:39,438
the sage on the stage,

22
00:00:39,438 --> 00:00:41,337
and the sleeping guy

23
00:00:41,337 --> 00:00:42,238
in the back. (Laughter)

24
00:00:42,238 --> 00:00:45,175
Just like today.

25
00:00:45,175 --> 00:00:48,286
So my co-teacher,

26
00:00:48,286 --> 00:00:49,926
Sebastian Thrun, and I thought,

27
00:00:49,926 --> 00:00:51,700
there must be a better way.

28
00:00:51,700 --> 00:00:53,354
We challenged ourselves

29
00:00:53,354 --> 00:00:54,645
to create an online class

30
00:00:54,645 --> 00:00:56,276
that would be equal or better

31
00:00:56,276 --> 00:00:58,612
in quality to our Stanford class,

32
00:00:58,612 --> 00:01:00,822
but to bring it to anyone

33
00:01:00,822 --> 00:01:02,452
in the world for free.

34
00:01:02,452 --> 00:01:05,020
We announced the class on July 29th,

35
00:01:05,020 --> 00:01:08,229
and within two weeks, 50,000 people

36
00:01:08,229 --> 00:01:09,636
had signed up for it.

37
00:01:09,636 --> 00:01:12,741
And that grew to 160,000 students

38
00:01:12,741 --> 00:01:14,805
from 209 countries.

39
00:01:14,805 --> 00:01:16,613
We were thrilled to have

40
00:01:16,613 --> 00:01:17,734
that kind of audience,

41
00:01:17,734 --> 00:01:20,373
and just a bit terrified that we

42
00:01:20,373 --> 00:01:22,684
hadn't finished preparing the class yet. (Laughter)

43
00:01:22,684 --> 00:01:24,296
So we got to work.

44
00:01:24,296 --> 00:01:25,932
We studied what others had done,

45
00:01:25,932 --> 00:01:28,255
what we could copy and what we could change.

46
00:01:28,255 --> 00:01:30,916
Benjamin Bloom had showed

47
00:01:30,916 --> 00:01:32,725
that one-on-one tutoring works best,

48
00:01:32,725 --> 00:01:34,853
so that's what we tried to emulate,

49
00:01:34,853 --> 00:01:36,293
like with me and my mom,

50
00:01:36,293 --> 00:01:37,798
even though we knew

51
00:01:37,798 --> 00:01:39,716
it would be one-on-thousands.

52
00:01:39,716 --> 00:01:41,917
Here, an overhead video camera

53
00:01:41,917 --> 00:01:43,749
is recording me as I'm talking

54
00:01:43,749 --> 00:01:45,428
and drawing on a piece of paper.

55
00:01:45,428 --> 00:01:47,837
A student said, "This class felt

56
00:01:47,837 --> 00:01:48,989
like sitting in a bar

57
00:01:48,989 --> 00:01:50,468
with a really smart friend

58
00:01:50,468 --> 00:01:51,672
who's explaining something

59
00:01:51,672 --> 00:01:53,973
you haven't grasped, but are about to."

60
00:01:53,973 --> 00:01:56,089
And that's exactly what we were aiming for.

61
00:01:56,089 --> 00:01:58,877
Now, from Khan Academy, we saw

62
00:01:58,877 --> 00:02:00,701
that short 10-minute videos

63
00:02:00,701 --> 00:02:02,477
worked much better than trying

64
00:02:02,477 --> 00:02:04,548
to record an hour-long lecture

65
00:02:04,548 --> 00:02:06,965
and put it on the small-format screen.

66
00:02:06,965 --> 00:02:09,157
We decided to go even shorter

67
00:02:09,157 --> 00:02:10,805
and more interactive.

68
00:02:10,805 --> 00:02:13,052
Our typical video is two minutes,

69
00:02:13,052 --> 00:02:14,804
sometimes shorter, never more

70
00:02:14,804 --> 00:02:17,493
than six, and then we pause for

71
00:02:17,493 --> 00:02:18,860
a quiz question, to make it

72
00:02:18,860 --> 00:02:20,738
feel like one-on-one tutoring.

73
00:02:20,738 --> 00:02:23,140
Here, I'm explaining how a computer uses

74
00:02:23,140 --> 00:02:24,388
the grammar of English

75
00:02:24,388 --> 00:02:26,565
to parse sentences, and here,

76
00:02:26,565 --> 00:02:28,629
there's a pause and the student

77
00:02:28,629 --> 00:02:30,866
has to reflect, understand what's going on

78
00:02:30,866 --> 00:02:32,506
and check the right boxes

79
00:02:32,506 --> 00:02:33,923
before they can continue.

80
00:02:33,923 --> 00:02:36,371
Students learn best when

81
00:02:36,371 --> 00:02:37,619
they're actively practicing.

82
00:02:37,619 --> 00:02:39,915
We wanted to engage them, to have them grapple

83
00:02:39,915 --> 00:02:42,883
with ambiguity and guide them to synthesize

84
00:02:42,883 --> 00:02:44,548
the key ideas themselves.

85
00:02:44,548 --> 00:02:46,260
We mostly avoid questions

86
00:02:46,260 --> 00:02:47,971
like, "Here's a formula, now

87
00:02:47,971 --> 00:02:49,046
tell me the value of Y

88
00:02:49,046 --> 00:02:50,258
when X is equal to two."

89
00:02:50,258 --> 00:02:52,075
We preferred open-ended questions.

90
00:02:52,075 --> 00:02:55,253
One student wrote, "Now I'm seeing

91
00:02:55,253 --> 00:02:57,115
Bayes networks and examples of

92
00:02:57,115 --> 00:02:58,683
game theory everywhere I look."

93
00:02:58,683 --> 00:03:00,421
And I like that kind of response.

94
00:03:00,421 --> 00:03:02,227
That's just what we were going for.

95
00:03:02,227 --> 00:03:04,421
We didn't want students to memorize the formulas;

96
00:03:04,421 --> 00:03:05,611
we wanted to change the way

97
00:03:05,611 --> 00:03:06,701
they looked at the world.

98
00:03:06,701 --> 00:03:08,027
And we succeeded.

99
00:03:08,027 --> 00:03:10,427
Or, I should say, the students succeeded.

100
00:03:10,427 --> 00:03:12,346
And it's a little bit ironic

101
00:03:12,346 --> 00:03:15,259
that we set about to disrupt traditional education,

102
00:03:15,259 --> 00:03:17,275
and in doing so, we ended up

103
00:03:17,275 --> 00:03:18,859
making our online class

104
00:03:18,859 --> 00:03:21,186
much more like a traditional college class

105
00:03:21,186 --> 00:03:23,027
than other online classes.

106
00:03:23,027 --> 00:03:26,242
Most online classes, the videos are always available.

107
00:03:26,242 --> 00:03:28,059
You can watch them any time you want.

108
00:03:28,059 --> 00:03:30,346
But if you can do it any time,

109
00:03:30,346 --> 00:03:31,851
that means you can do it tomorrow,

110
00:03:31,851 --> 00:03:33,259
and if you can do it tomorrow,

111
00:03:33,259 --> 00:03:35,421
well, you may not ever

112
00:03:35,421 --> 00:03:37,179
get around to it. (Laughter)

113
00:03:37,179 --> 00:03:39,371
So we brought back the innovation

114
00:03:39,371 --> 00:03:41,458
of having due dates. (Laughter)

115
00:03:41,458 --> 00:03:42,770
You could watch the videos

116
00:03:42,770 --> 00:03:44,682
any time you wanted during the week,

117
00:03:44,682 --> 00:03:45,837
but at the end of the week,

118
00:03:45,837 --> 00:03:47,554
you had to get the homework done.

119
00:03:47,554 --> 00:03:49,467
This motivated the students to keep going, and it also

120
00:03:49,467 --> 00:03:52,323
meant that everybody was working

121
00:03:52,323 --> 00:03:53,841
on the same thing at the same time,

122
00:03:53,841 --> 00:03:55,338
so if you went into a discussion forum,

123
00:03:55,338 --> 00:03:58,042
you could get an answer from a peer within minutes.

124
00:03:58,042 --> 00:04:00,981
Now, I'll show you some of the forums, most of which

125
00:04:00,981 --> 00:04:03,747
were self-organized by the students themselves.

126
00:04:03,747 --> 00:04:06,971
From Daphne Koller and Andrew Ng, we learned

127
00:04:06,971 --> 00:04:08,967
the concept of "flipping" the classroom.

128
00:04:08,967 --> 00:04:10,262
Students watched the videos

129
00:04:10,262 --> 00:04:11,850
on their own, and then they

130
00:04:11,850 --> 00:04:13,560
come together to discuss them.

131
00:04:13,560 --> 00:04:16,717
From Eric Mazur, I learned about peer instruction,

132
00:04:16,717 --> 00:04:19,341
that peers can be the best teachers,

133
00:04:19,341 --> 00:04:20,773
because they're the ones

134
00:04:20,773 --> 00:04:23,621
that remember what it's like to not understand.

135
00:04:23,621 --> 00:04:26,302
Sebastian and I have forgotten some of that.

136
00:04:26,302 --> 00:04:28,701
Of course, we couldn't have

137
00:04:28,701 --> 00:04:30,333
a classroom discussion with

138
00:04:30,333 --> 00:04:31,879
tens of thousands of students,

139
00:04:31,879 --> 00:04:35,365
so we encouraged and nurtured these online forums.

140
00:04:35,365 --> 00:04:38,413
And finally, from Teach For America,

141
00:04:38,413 --> 00:04:39,894
I learned that a class is not

142
00:04:39,894 --> 00:04:41,390
primarily about information.

143
00:04:41,390 --> 00:04:44,221
More important is motivation and determination.

144
00:04:44,221 --> 00:04:46,061
It was crucial that the students see

145
00:04:46,061 --> 00:04:47,861
that we're working hard for them and

146
00:04:47,861 --> 00:04:49,270
they're all supporting each other.

147
00:04:49,270 --> 00:04:52,238
Now, the class ran 10 weeks,

148
00:04:52,238 --> 00:04:56,149
and in the end, about half of the 160,000 students watched

149
00:04:56,149 --> 00:04:57,712
at least one video each week,

150
00:04:57,712 --> 00:05:00,405
and over 20,000 finished all the homework,

151
00:05:00,405 --> 00:05:02,069
putting in 50 to 100 hours.

152
00:05:02,069 --> 00:05:03,622
They got this statement of accomplishment.

153
00:05:03,622 --> 00:05:05,741
So what have we learned?

154
00:05:05,741 --> 00:05:08,557
Well, we tried some old ideas

155
00:05:08,557 --> 00:05:10,214
and some new and put them together,

156
00:05:10,214 --> 00:05:12,296
but there are more ideas to try.

157
00:05:12,296 --> 00:05:14,110
Sebastian's teaching another class now.

158
00:05:14,110 --> 00:05:15,478
I'll do one in the fall.

159
00:05:15,478 --> 00:05:19,238
Stanford Coursera, Udacity, MITx

160
00:05:19,238 --> 00:05:21,509
and others have more classes coming.

161
00:05:21,509 --> 00:05:22,997
It's a really exciting time.

162
00:05:22,997 --> 00:05:24,469
But to me, the most exciting

163
00:05:24,469 --> 00:05:27,445
part of it is the data that we're gathering.

164
00:05:27,445 --> 00:05:30,141
We're gathering thousands

165
00:05:30,141 --> 00:05:31,797
of interactions per student per class,

166
00:05:31,797 --> 00:05:34,005
billions of interactions altogether,

167
00:05:34,005 --> 00:05:36,509
and now we can start analyzing that,

168
00:05:36,509 --> 00:05:37,973
and when we learn from that,

169
00:05:37,973 --> 00:05:39,214
do experimentations,

170
00:05:39,214 --> 00:05:41,454
that's when the real revolution will come.

171
00:05:41,454 --> 00:05:44,230
And you'll be able to see the results from

172
00:05:44,230 --> 00:05:46,493
a new generation of amazing students.

173
00:05:46,493 --> 00:05:48,746
(Applause)