1
00:00:00,390 --> 00:00:03,140
I've talked about one very impressive
study by John Hattie.

2
00:00:03,140 --> 00:00:07,040
He collected data for years where each of
his data points

3
00:00:07,040 --> 00:00:10,630
was itself the result of another study
that took a long time.

4
00:00:10,630 --> 00:00:11,320
That's a lot of work.

5
00:00:12,630 --> 00:00:14,730
I want to explain to you how MOOCs offer
the

6
00:00:14,730 --> 00:00:18,410
opportunity to do all these studies much
more quickly and efficiently.

7
00:00:19,640 --> 00:00:23,510
The first thing to know is every single
click on the major platforms is tracked.

8
00:00:24,850 --> 00:00:27,220
If you were to pause the video now, they
would know.

9
00:00:27,220 --> 00:00:30,720
If you are watching this accelerated, they
know it as well.

10
00:00:30,720 --> 00:00:34,190
They also know how often a student posts
on the

11
00:00:34,190 --> 00:00:36,860
forum or how many time they try to post
questions.

12
00:00:38,130 --> 00:00:40,850
All these days I analysed, some of the
smartest brain

13
00:00:40,850 --> 00:00:42,950
in the world, actually try to make sense
out of it.

14
00:00:44,460 --> 00:00:47,620
The question, the big question is, what
works best for teaching online?

15
00:00:47,620 --> 00:00:51,890
And maybe even what works best for
teaching?

16
00:00:51,890 --> 00:00:54,220
A video of a talking head, beamer slides
with

17
00:00:54,220 --> 00:00:58,790
an insert of the instructor, or maybe
without the instructor.

18
00:00:58,790 --> 00:01:00,240
Some guy's paper cutouts.

19
00:01:01,370 --> 00:01:01,610
Who knows?

20
00:01:01,610 --> 00:01:05,450
This data is analysed, and it's not a
sample size of n equals 20.

21
00:01:05,450 --> 00:01:07,170
But, it's more like n equals two million.

22
00:01:08,850 --> 00:01:11,110
I'm tried to give you a flavor of what is
already possible.

23
00:01:12,220 --> 00:01:14,350
Let's start with a concrete situation.

24
00:01:14,350 --> 00:01:17,410
Let's say you have an ad for a cool
project to change the world.

25
00:01:17,410 --> 00:01:19,190
You want people to help you.

26
00:01:19,190 --> 00:01:22,220
So you write a flyer that say, what to
change the world.

27
00:01:22,220 --> 00:01:23,690
Come and join me at this day and place.

28
00:01:25,090 --> 00:01:26,910
You know that that's not enough to attract
people.

29
00:01:26,910 --> 00:01:28,680
So you want to had, to add a hook before.

30
00:01:28,680 --> 00:01:32,740
You don't want, you don't know for sure
what to put in.

31
00:01:32,740 --> 00:01:38,480
You think maybe, the line, want cookies,
or, want pizza, might attract people.

32
00:01:38,480 --> 00:01:43,040
But what will grab people's attention and
attract people to your purchase/?

33
00:01:43,040 --> 00:01:45,910
You don't know, And, in the real world,
you wont know.

34
00:01:45,910 --> 00:01:47,900
Unless you try both in parallel, and

35
00:01:47,900 --> 00:01:51,300
see afterwards which one is most suc-,
successful.

36
00:01:51,300 --> 00:01:52,230
How do you do that?

37
00:01:52,230 --> 00:01:54,960
Well, by interviewing the people who show
up.

38
00:01:54,960 --> 00:01:56,960
This takes time ,and again, it doesn't
scale well.

39
00:01:58,350 --> 00:02:00,710
Let's say now your Google, you'll
literally

40
00:02:00,710 --> 00:02:03,680
make millions of dollars of revenue per
hour.

41
00:02:03,680 --> 00:02:06,650
Most of it through people clicking on ads
in the sidebar.

42
00:02:08,009 --> 00:02:09,419
One of your employees thinks that a

43
00:02:09,419 --> 00:02:12,220
particular shade of green will definitely
help

44
00:02:12,220 --> 00:02:16,260
highlight the ads, but not too much so
that it doesn't bother the users.

45
00:02:17,610 --> 00:02:21,990
You as Google want to try it, because you
know there is a lot of money there.

46
00:02:21,990 --> 00:02:24,840
But you are afraid of changing something
that works pretty well already.

47
00:02:24,840 --> 00:02:28,710
So what you can do is to pick a random 10%
of the users

48
00:02:28,710 --> 00:02:34,750
of a random small country, say, Belgium,
and show to them that new green shape.

49
00:02:35,810 --> 00:02:37,840
And then you can compare those users to

50
00:02:37,840 --> 00:02:40,110
other users, where the experiment was not
performed.

51
00:02:41,140 --> 00:02:42,830
Maybe you can also pick another country.

52
00:02:42,830 --> 00:02:48,370
Say, the Netherlands, and do the same
experiment with another color, say orange.

53
00:02:48,370 --> 00:02:51,466
Or you can try changing the size of the
ads, or all at

54
00:02:51,466 --> 00:02:53,770
the same time, fitting each user into

55
00:02:53,770 --> 00:02:57,282
their own highly individualized blend of
experiments.

56
00:02:57,282 --> 00:03:00,690
As long as you can extract meaningful
data, why wouldn't you do it?

57
00:03:01,840 --> 00:03:03,230
Well that's what Google does.

58
00:03:03,230 --> 00:03:07,720
Every time you go on Google hundreds of
experiments are actually performed on you.

59
00:03:07,720 --> 00:03:11,014
The page you get for a particular search,
is slightly different

60
00:03:11,014 --> 00:03:14,890
from the page offered to anyone else for
the same search.

61
00:03:14,890 --> 00:03:16,246
And they try to optimize that page for
you.

62
00:03:16,246 --> 00:03:22,160
This is called A/B testing, and it's used
extensively to optimize the web.

63
00:03:23,280 --> 00:03:24,960
I know it sounds silly to talk about this

64
00:03:24,960 --> 00:03:27,280
for education but I don't think it really
is silly.

65
00:03:28,740 --> 00:03:32,590
Many blackboards are dyed green because
some tests were performed a long

66
00:03:32,590 --> 00:03:35,510
time ago and people figure out this was
better for the eyes.

67
00:03:35,510 --> 00:03:40,000
You can see the similarity with Google's
experiment there.

68
00:03:40,000 --> 00:03:41,650
Many more studies have been done since

69
00:03:41,650 --> 00:03:44,380
in education to try and improve other
factors.

70
00:03:44,380 --> 00:03:46,600
My be more meaningful than just a
blackboard color.

71
00:03:46,600 --> 00:03:49,890
But this is slow and requires lots of
manual work.

72
00:03:51,010 --> 00:03:54,200
Again, similar studies can be done in a
MOOC space, but

73
00:03:54,200 --> 00:03:57,390
actually much more is possible, much
faster, and much more efficiently.

74
00:03:59,020 --> 00:04:01,060
For instance, you can alter the material

75
00:04:01,060 --> 00:04:03,590
presented to the students in very subtle
ways.

76
00:04:03,590 --> 00:04:05,330
And see what has an effect and what does
not.

77
00:04:06,560 --> 00:04:08,810
So let's say you split your class in

78
00:04:08,810 --> 00:04:11,960
two, and measured how effective it
actually is to

79
00:04:11,960 --> 00:04:17,180
start a class with a little preview of the
problem that you want to solve at the end.

80
00:04:17,180 --> 00:04:21,370
This could be effortless by the teacher,
or someone doing educational research.

81
00:04:22,930 --> 00:04:26,390
The technology to perform these studies
exists, and is well understood.

82
00:04:26,390 --> 00:04:28,690
Because it was used extensively in other
industries

83
00:04:28,690 --> 00:04:31,180
before, and is now being applied to
education.

84
00:04:32,420 --> 00:04:34,870
Some conferences are starting on these
topics and releasing

85
00:04:34,870 --> 00:04:38,590
lots of insight into what works and what
does not.

86
00:04:38,590 --> 00:04:43,560
It's hard to argue when someone comes with
a study conducted on millions of students.

87
00:04:43,560 --> 00:04:48,890
You ask for a follow up and six months
later, they come back with that follow up.

88
00:04:48,890 --> 00:04:52,020
This is tremendously accelerating for
education, and very exciting.