WEBVTT 00:00:00.763 --> 00:00:02.908 As I said a lot of new technology is being built. 00:00:03.515 --> 00:00:05.490 But it's very easy to be distracted by this technology. 00:00:06.396 --> 00:00:09.623 It's easy to forget what to focus on because of all the shiny visuals. 00:00:10.677 --> 00:00:12.747 Simultaneously technology can open new doors 00:00:13.428 --> 00:00:15.532 or put in new constraints on our pedagogy. 00:00:16.160 --> 00:00:17.694 So let's think again, let's refocus. 00:00:18.809 --> 00:00:20.909 When we teach what should we care about the most? 00:00:22.319 --> 00:00:23.865 While I have some practical experience as an instructor, 00:00:25.159 --> 00:00:26.842 but I am not an expert in education. 00:00:27.669 --> 00:00:29.940 So I rely on an influential study by John Hattie 00:00:31.949 --> 00:00:34.611 He looked at 800 meta analyses and synthesized them. 00:00:35.686 --> 00:00:37.164 He was looking for what influences learning, 00:00:37.740 --> 00:00:40.242 which circumstances, tools and techniques had 00:00:40.406 --> 00:00:43.586 the best effects, or which just had average effects. 00:00:44.695 --> 00:00:48.026 So let's pick and choose a few of those factors in no order at the moment. 00:00:49.398 --> 00:00:50.909 The first is physical attributes. 00:00:52.004 --> 00:00:53.845 The class size, for instance. 00:00:54.423 --> 00:00:56.355 It's clear that this is relevant in a MOOC setting. 00:00:57.492 --> 00:01:00.433 The second factor is computer-assisted instruction. 00:01:01.049 --> 00:01:03.003 It's also clear that this is crucial in a MOOC 00:01:04.351 --> 00:01:06.468 There are other factors that will affect teaching. 00:01:07.323 --> 00:01:09.035 Things that are maybe more intrinsic to the student. 00:01:10.268 --> 00:01:13.165 Course, we would like to think that we can teach anyone, but we expect the 00:01:13.200 --> 00:01:16.590 students' disposition to learn, their intrinsic motivation, to 00:01:16.868 --> 00:01:18.285 have to be a factor in the outcome. 00:01:19.456 --> 00:01:23.148 In some way, MOOCs have a self-selected audience for now so this is mitigated. 00:01:24.892 --> 00:01:27.562 There's also the factor of the student's prior cognitive ability. 00:01:28.825 --> 00:01:30.550 Think of something like their IQ, for instance. 00:01:31.995 --> 00:01:33.160 We have little control there. 00:01:34.565 --> 00:01:37.031 Fortunately, there are parts we can actively change. 00:01:38.318 --> 00:01:40.712 The instructional quality of the course, for instance. 00:01:41.695 --> 00:01:45.252 That is, the student's perception of the quality of the instruction they get, 00:01:45.957 --> 00:01:47.226 according to Hattie's definition. 00:01:48.535 --> 00:01:50.058 What about individualization? 00:01:50.648 --> 00:01:52.896 The design of a learning program specific to the student. 00:01:54.230 --> 00:01:55.356 Or maybe mastery learning. 00:01:56.156 --> 00:01:57.460 This is when students are given tests 00:01:58.035 --> 00:02:00.571 on relatively easy material soon after instruction. 00:02:01.602 --> 00:02:04.710 And if it seems necessary they're given more work again. 00:02:04.710 --> 00:02:06.480 They're retested until they get it. 00:02:08.340 --> 00:02:11.070 Maybe a teacher could challenge students by outlining goals, 00:02:12.868 --> 00:02:16.079 but there is also feedback, which in Hattie's mind, includes 00:02:16.482 --> 00:02:21.029 positive reinforcement for what has been done well, information about 00:02:21.308 --> 00:02:25.746 what needs to improve but also includes merely clarifying the goals, 00:02:27.272 --> 00:02:28.254 as part of the feedback. 00:02:30.095 --> 00:02:31.601 There's also peer tutoring, where the 00:02:31.873 --> 00:02:33.778 students explain, check, and assess each other. 00:02:35.122 --> 00:02:36.060 How about a good old test? 00:02:36.574 --> 00:02:39.444 How about external factors, such as the presence of a television at home? 00:02:40.843 --> 00:02:43.796 Hattie identified 130 different such factors. 00:02:45.236 --> 00:02:46.172 I only picked a few here. 00:02:47.002 --> 00:02:49.519 But it's already a lot of influencers on the student's achievement. 00:02:51.383 --> 00:02:53.346 But how do you rank them so we could make 00:02:53.426 --> 00:02:55.988 sense of the potential thug of that large number of influencers [sic: influences]? 00:02:58.566 --> 00:02:59.814 So what do you think ranked the best? 00:03:00.420 --> 00:03:01.039 What ranked the worst? 00:03:02.433 --> 00:03:04.586 You can maybe guess that television is the worst. 00:03:05.137 --> 00:03:07.680 That seems kind of an obvious candidate. 00:03:08.395 --> 00:03:11.126 In fact, television slowed down a student's progress. 00:03:12.485 --> 00:03:15.542 Next up, and very controversially, was class size. 00:03:16.907 --> 00:03:20.122 This is still very much at the bottom; in fact the data indicates 00:03:21.033 --> 00:03:23.310 inconclusively that small class sizes have 00:03:23.310 --> 00:03:26.855 a tiny positive effect on the student's learning. 00:03:28.431 --> 00:03:30.000 The best way to make sense of this is probably 00:03:30.360 --> 00:03:32.038 to better understand Hattie's methodology. 00:03:33.400 --> 00:03:36.178 This result comes after accounting for all 00:03:36.359 --> 00:03:39.514 the other effects, if all else remains equal. 00:03:42.180 --> 00:03:44.172 So if a politician somewhere bases a decision on this study, and 00:03:44.431 --> 00:03:47.579 blindly doubles the class size of all the classrooms in their country - 00:03:48.590 --> 00:03:51.070 presumably, importantly, by firing half of the teachers - 00:03:52.111 --> 00:03:54.517 well the teachers that remain will suddenly be swamped with work. 00:03:55.290 --> 00:03:57.043 And unable to do some of the other 00:03:57.301 --> 00:03:59.295 techniques that actually have a very positive effect. 00:04:00.286 --> 00:04:01.157 So this would be bad. 00:04:03.186 --> 00:04:04.409 Now on to the positive effects. 00:04:05.681 --> 00:04:07.723 When Hattie performed this study he very well understood 00:04:08.177 --> 00:04:10.282 that most of those techniques had a positive effect. 00:04:11.098 --> 00:04:13.314 The point was to identify those that were MOST effective. 00:04:14.469 --> 00:04:16.544 So he set the bar at the average effect size. 00:04:17.631 --> 00:04:21.083 Obviously we're interested in the top effects, those way above the bar. 00:04:22.014 --> 00:04:23.597 But we should proceed methodically. 00:04:24.459 --> 00:04:26.083 Below that bar, still with a positive effect 00:04:26.546 --> 00:04:28.965 but not so great, are computer assisted instruction, 00:04:30.235 --> 00:04:31.715 individualization, and testing. 00:04:33.990 --> 00:04:35.679 Above that, and in a position that starts to be 00:04:35.868 --> 00:04:39.376 very useful, are challenging students with goal [sic: goals], 00:04:39.723 --> 00:04:42.271 peer tutoring, mastery learning, and, slightly 00:04:42.512 --> 00:04:45.077 above all those three, the student's disposition to learn. 00:04:46.877 --> 00:04:49.248 But the three stellar effects are feedback, 00:04:49.862 --> 00:04:51.597 student's prior cognitive ability, 00:04:52.094 --> 00:04:53.164 and instructional quality. 00:04:54.355 --> 00:04:58.085 Each is at a level where you expect a 50% increase in the pace of learning. 00:04:59.382 --> 00:05:01.416 Of course feedback is a bit of an umbrella term here. 00:05:02.562 --> 00:05:06.235 It covers pure feedback or mastery learning, for instance. 00:05:07.753 --> 00:05:09.874 Note, also, that computer-assisted goes up 00:05:10.470 --> 00:05:15.724 as the course becomes more interactive and more engaging. And also that smaller 00:05:16.061 --> 00:05:18.371 effects can be combined to accumulate traction. 00:05:18.963 --> 00:05:20.275 So nothing should be fully dismissed. 00:05:21.875 --> 00:05:25.198 So, we now have Hattie's final ranking for the effects I've selected. 00:05:26.888 --> 00:05:28.031 Why am I talking about this? 00:05:28.679 --> 00:05:30.876 Well, because in MOOCs some of it is especially relevant. 00:05:32.080 --> 00:05:34.542 Mastery learning has been trumpeted again and again for MOOCs. 00:05:35.085 --> 00:05:38.944 You can offer a quiz, sampling questions and answers until the student gets it. 00:05:40.131 --> 00:05:43.424 Interaction as well. More and more MOOCs have interactive components 00:05:43.860 --> 00:05:46.124 that students can use to visually see what they are learning. 00:05:47.461 --> 00:05:49.172 So this is all a success story for MOOCs 00:05:49.491 --> 00:05:52.006 integrating fully those effects in the format, if you want. 00:05:53.499 --> 00:05:56.508 Peer effects are also important and also present in MOOCS. 00:05:57.282 --> 00:05:58.343 But they're not fully mastered yet. 00:05:59.227 --> 00:06:01.647 There are some structured peer grading components in MOOCS 00:06:02.239 --> 00:06:04.150 but they're still in early stages and can be improved. 00:06:05.208 --> 00:06:07.714 Also, few people know for sure how to get a forum started, 00:06:08.448 --> 00:06:11.664 one that encourages students from all backgrounds to engage with their peers. 00:06:12.623 --> 00:06:14.147 I'm certainly not one of those experts. 00:06:14.855 --> 00:06:17.312 It's a skill, something that needs to be learned and practiced. 00:06:18.433 --> 00:06:21.210 Some of those experts, on the other hand, might be lurking in this class. 00:06:22.214 --> 00:06:25.552 So please help the discussion get started on the forums, if you feel you can help. 00:06:27.115 --> 00:06:28.376 I would love to engage with all of you. 00:06:29.141 --> 00:06:30.259 Share your opinion, don't be afraid. 00:06:31.060 --> 00:06:32.239 There's definitely no right or wrong. 00:06:33.032 --> 00:06:34.754 Even Hattie's study is open to criticism. 00:06:35.632 --> 00:06:36.913 And especially my retelling of it. 00:06:38.028 --> 00:06:39.820 This is education. There is no one answer. 00:06:40.633 --> 00:06:43.043 So, I would love to have a discussion on the forums about all that. 00:06:44.394 --> 00:06:46.944 The last effect I want to discuss, that will be the 00:06:47.217 --> 00:06:50.978 next video, is the most important, according to Hattie: that is "feedback".