0:00:00.763,0:00:02.908
As I said a lot of new technology is being[br]built.

0:00:03.515,0:00:05.490
But it's very easy to be distracted by[br]this technology.

0:00:06.396,0:00:09.623
It's easy to forget what to focus on[br]because of all the shiny visuals.

0:00:10.677,0:00:12.747
Simultaneously technology can open new[br]doors

0:00:13.428,0:00:15.532
or put in new constraints on our pedagogy.

0:00:16.160,0:00:17.694
So let's think again, let's refocus.

0:00:18.809,0:00:20.909
When we teach what should we care about[br]the most?

0:00:22.319,0:00:23.865
While I have some practical experience as an instructor,

0:00:25.159,0:00:26.842
but I am not an expert in education.

0:00:27.669,0:00:29.940
So I rely on an influential study by John[br]Hattie

0:00:31.949,0:00:34.611
He looked at 800 meta analyses and[br]synthesized them.

0:00:35.686,0:00:37.164
He was looking for what influences[br]learning,

0:00:37.740,0:00:40.242
which circumstances, tools and techniques[br]had

0:00:40.406,0:00:43.586
the best effects, or which just had average[br]effects.

0:00:44.695,0:00:48.026
So let's pick and choose a few of those[br]factors in no order at the moment.

0:00:49.398,0:00:50.909
The first is physical attributes.

0:00:52.004,0:00:53.845
The class size, for instance.

0:00:54.423,0:00:56.355
It's clear that this is relevant in a MOOC[br]setting.

0:00:57.492,0:01:00.433
The second factor is computer-assisted[br]instruction.

0:01:01.049,0:01:03.003
It's also clear that this is crucial in a MOOC

0:01:04.351,0:01:06.468
There are other factors that will affect[br]teaching.

0:01:07.323,0:01:09.035
Things that are maybe more intrinsic to[br]the student.

0:01:10.268,0:01:13.165
Course, we would like to think that we can[br]teach anyone, but we expect the

0:01:13.200,0:01:16.590
students' disposition to learn, their[br]intrinsic motivation, to

0:01:16.868,0:01:18.285
have to be a factor in the outcome.

0:01:19.456,0:01:23.148
In some way, MOOCs have a self-selected[br]audience for now so this is mitigated.

0:01:24.892,0:01:27.562
There's also the factor of the student's[br]prior cognitive ability.

0:01:28.825,0:01:30.550
Think of something like their IQ, for[br]instance.

0:01:31.995,0:01:33.160
We have little control there.

0:01:34.565,0:01:37.031
Fortunately, there are parts we can[br]actively change.

0:01:38.318,0:01:40.712
The instructional quality of the course,[br]for instance.

0:01:41.695,0:01:45.252
That is, the student's perception of the[br]quality of the instruction they get,

0:01:45.957,0:01:47.226
according to Hattie's definition.

0:01:48.535,0:01:50.058
What about individualization?

0:01:50.648,0:01:52.896
The design of a learning program[br]specific to the student.

0:01:54.230,0:01:55.356
Or maybe mastery learning.

0:01:56.156,0:01:57.460
This is when students are given tests

0:01:58.035,0:02:00.571
on relatively easy material soon after[br]instruction.

0:02:01.602,0:02:04.710
And if it seems necessary they're given more work again. [br]

0:02:04.710,0:02:06.480
They're retested until they get it.

0:02:08.340,0:02:11.070
Maybe a teacher could challenge students[br]by outlining goals,

0:02:12.868,0:02:16.079
but there is also feedback, which in Hattie's[br]mind, includes

0:02:16.482,0:02:21.029
positive reinforcement for what has been[br]done well, information about

0:02:21.308,0:02:25.746
what needs to improve but also includes merely clarifying the goals,

0:02:27.272,0:02:28.254
as part of the feedback.

0:02:30.095,0:02:31.601
There's also peer tutoring, where the

0:02:31.873,0:02:33.778
students explain, check, and assess each[br]other.

0:02:35.122,0:02:36.060
How about a good old test?

0:02:36.574,0:02:39.444
How about external factors, such as the[br]presence of a television at home?

0:02:40.843,0:02:43.796
Hattie identified 130 different such[br]factors.

0:02:45.236,0:02:46.172
I only picked a few here.

0:02:47.002,0:02:49.519
But it's already a lot of influencers on[br]the student's achievement.

0:02:51.383,0:02:53.346
But how do you rank them so we could make

0:02:53.426,0:02:55.988
sense of the potential thug of that large[br]number of influencers [sic: influences]?

0:02:58.566,0:02:59.814
So what do you think ranked the best?

0:03:00.420,0:03:01.039
What ranked the worst?

0:03:02.433,0:03:04.586
You can maybe guess that television is the[br]worst.

0:03:05.137,0:03:07.680
That seems kind of an obvious candidate.

0:03:08.395,0:03:11.126
In fact, television slowed down a student's progress.

0:03:12.485,0:03:15.542
Next up, and very controversially, was class[br]size.

0:03:16.907,0:03:20.122
This is still very much at the bottom; in[br]fact the data indicates

0:03:21.033,0:03:23.310
inconclusively that small class sizes have[br]

0:03:23.310,0:03:26.855
a tiny positive effect on the student's [br]learning.

0:03:28.431,0:03:30.000
The best way to make sense of this is probably

0:03:30.360,0:03:32.038
to better understand Hattie's methodology.

0:03:33.400,0:03:36.178
This result comes after accounting for all

0:03:36.359,0:03:39.514
the other effects, if all else remains[br]equal.

0:03:42.180,0:03:44.172
So if a politician somewhere bases a[br]decision on this study, and

0:03:44.431,0:03:47.579
blindly doubles the class size of all the[br]classrooms in their country -

0:03:48.590,0:03:51.070
presumably, importantly, by firing half of the[br]teachers -

0:03:52.111,0:03:54.517
well the teachers that remain will suddenly be swamped with work.

0:03:55.290,0:03:57.043
And unable to do some of the other

0:03:57.301,0:03:59.295
techniques that actually have a very[br]positive effect.

0:04:00.286,0:04:01.157
So this would be bad.

0:04:03.186,0:04:04.409
Now on to the positive effects.

0:04:05.681,0:04:07.723
When Hattie performed this study he very[br]well understood

0:04:08.177,0:04:10.282
that most of those techniques had a[br]positive effect.

0:04:11.098,0:04:13.314
The point was to identify those that were[br]MOST effective.

0:04:14.469,0:04:16.544
So he set the bar at the average effect[br]size.

0:04:17.631,0:04:21.083
Obviously we're interested in the top[br]effects, those way above the bar.

0:04:22.014,0:04:23.597
But we should proceed methodically.

0:04:24.459,0:04:26.083
Below that bar, still with a positive[br]effect

0:04:26.546,0:04:28.965
but not so great, are computer assisted[br]instruction,

0:04:30.235,0:04:31.715
individualization, and testing.

0:04:33.990,0:04:35.679
Above that, and in a position that starts[br]to be

0:04:35.868,0:04:39.376
very useful, are challenging students with[br]goal [sic: goals],

0:04:39.723,0:04:42.271
peer tutoring, mastery learning, and, slightly

0:04:42.512,0:04:45.077
above all those three, the student's disposition to learn.

0:04:46.877,0:04:49.248
But the three stellar effects are feedback,

0:04:49.862,0:04:51.597
student's prior cognitive ability,

0:04:52.094,0:04:53.164
and instructional quality.

0:04:54.355,0:04:58.085
Each is at a level where you expect a 50%[br]increase in the pace of learning.

0:04:59.382,0:05:01.416
Of course feedback is a bit of an umbrella[br]term here.

0:05:02.562,0:05:06.235
It covers pure feedback or mastery[br]learning, for instance.

0:05:07.753,0:05:09.874
Note, also, that computer-assisted goes up

0:05:10.470,0:05:15.724
as the course becomes more interactive[br]and more engaging. And also that smaller

0:05:16.061,0:05:18.371
effects can be combined to accumulate traction.

0:05:18.963,0:05:20.275
So nothing should be fully dismissed.

0:05:21.875,0:05:25.198
So, we now have Hattie's final ranking for[br]the effects I've selected.

0:05:26.888,0:05:28.031
Why am I talking about this?

0:05:28.679,0:05:30.876
Well, because in MOOCs some of it is[br]especially relevant.

0:05:32.080,0:05:34.542
Mastery learning has been trumpeted again and again for MOOCs.[br]

0:05:35.085,0:05:38.944
You can offer a quiz, sampling questions and answers until the student gets it.[br]

0:05:40.131,0:05:43.424
Interaction as well. More and more MOOCs have interactive components [br]

0:05:43.860,0:05:46.124
that students can use to visually see what[br]they are learning.

0:05:47.461,0:05:49.172
So this is all a success story for MOOCs

0:05:49.491,0:05:52.006
integrating fully those effects in the[br]format, if you want.

0:05:53.499,0:05:56.508
Peer effects are also important and also[br]present in MOOCS.

0:05:57.282,0:05:58.343
But they're not fully mastered yet.

0:05:59.227,0:06:01.647
There are some structured peer grading[br]components in MOOCS

0:06:02.239,0:06:04.150
but they're still in early stages and can[br]be improved.

0:06:05.208,0:06:07.714
Also, few people know for sure how to get[br]a forum started,

0:06:08.448,0:06:11.664
one that encourages students from all[br]backgrounds to engage with their peers.

0:06:12.623,0:06:14.147
I'm certainly not one of those[br]experts.

0:06:14.855,0:06:17.312
It's a skill, something that needs to be[br]learned and practiced.

0:06:18.433,0:06:21.210
Some of those experts, on the other hand,[br]might be lurking in this class.

0:06:22.214,0:06:25.552
So please help the discussion get started[br]on the forums, if you feel you can help.

0:06:27.115,0:06:28.376
I would love to engage with all of you.

0:06:29.141,0:06:30.259
Share your opinion, don't be afraid.

0:06:31.060,0:06:32.239
There's definitely no right or wrong.

0:06:33.032,0:06:34.754
Even Hattie's study is open to criticism.

0:06:35.632,0:06:36.913
And especially my retelling of it.

0:06:38.028,0:06:39.820
This is education. There is no one answer.

0:06:40.633,0:06:43.043
So, I would love to have a discussion on[br]the forums about all that.

0:06:44.394,0:06:46.944
The last effect I want to discuss, that[br]will be the

0:06:47.217,0:06:50.978
next video, is the most important, according[br]to Hattie: that is "feedback".