WEBVTT 00:00:00.390 --> 00:00:03.140 I've talked about one very impressive study by John Hattie. 00:00:03.140 --> 00:00:07.040 He collected data for years where each of his data points 00:00:07.040 --> 00:00:10.630 was itself the result of another study that took a long time. 00:00:10.630 --> 00:00:11.320 That's a lot of work. 00:00:12.630 --> 00:00:14.730 I want to explain to you how MOOCs offer the 00:00:14.730 --> 00:00:18.410 opportunity to do all these studies much more quickly and efficiently. 00:00:19.640 --> 00:00:23.510 The first thing to know is every single click on the major platforms is tracked. 00:00:24.850 --> 00:00:27.220 If you were to pause the video now, they would know. 00:00:27.220 --> 00:00:30.720 If you are watching this accelerated, they know it as well. 00:00:30.720 --> 00:00:34.190 They also know how often a student posts on the 00:00:34.190 --> 00:00:36.860 forum or how many time they try to post questions. 00:00:38.130 --> 00:00:40.850 All these days I analysed, some of the smartest brain 00:00:40.850 --> 00:00:42.950 in the world, actually try to make sense out of it. 00:00:44.460 --> 00:00:47.620 The question, the big question is, what works best for teaching online? 00:00:47.620 --> 00:00:51.890 And maybe even what works best for teaching? 00:00:51.890 --> 00:00:54.220 A video of a talking head, beamer slides with 00:00:54.220 --> 00:00:58.790 an insert of the instructor, or maybe without the instructor. 00:00:58.790 --> 00:01:00.240 Some guy's paper cutouts. 00:01:01.370 --> 00:01:01.610 Who knows? 00:01:01.610 --> 00:01:05.450 This data is analysed, and it's not a sample size of n equals 20. 00:01:05.450 --> 00:01:07.170 But, it's more like n equals two million. 00:01:08.850 --> 00:01:11.110 I'm tried to give you a flavor of what is already possible. 00:01:12.220 --> 00:01:14.350 Let's start with a concrete situation. 00:01:14.350 --> 00:01:17.410 Let's say you have an ad for a cool project to change the world. 00:01:17.410 --> 00:01:19.190 You want people to help you. 00:01:19.190 --> 00:01:22.220 So you write a flyer that say, what to change the world. 00:01:22.220 --> 00:01:23.690 Come and join me at this day and place. 00:01:25.090 --> 00:01:26.910 You know that that's not enough to attract people. 00:01:26.910 --> 00:01:28.680 So you want to had, to add a hook before. 00:01:28.680 --> 00:01:32.740 You don't want, you don't know for sure what to put in. 00:01:32.740 --> 00:01:38.480 You think maybe, the line, want cookies, or, want pizza, might attract people. 00:01:38.480 --> 00:01:43.040 But what will grab people's attention and attract people to your purchase/? 00:01:43.040 --> 00:01:45.910 You don't know, And, in the real world, you wont know. 00:01:45.910 --> 00:01:47.900 Unless you try both in parallel, and 00:01:47.900 --> 00:01:51.300 see afterwards which one is most suc-, successful. 00:01:51.300 --> 00:01:52.230 How do you do that? 00:01:52.230 --> 00:01:54.960 Well, by interviewing the people who show up. 00:01:54.960 --> 00:01:56.960 This takes time ,and again, it doesn't scale well. 00:01:58.350 --> 00:02:00.710 Let's say now your Google, you'll literally 00:02:00.710 --> 00:02:03.680 make millions of dollars of revenue per hour. 00:02:03.680 --> 00:02:06.650 Most of it through people clicking on ads in the sidebar. 00:02:08.009 --> 00:02:09.419 One of your employees thinks that a 00:02:09.419 --> 00:02:12.220 particular shade of green will definitely help 00:02:12.220 --> 00:02:16.260 highlight the ads, but not too much so that it doesn't bother the users. 00:02:17.610 --> 00:02:21.990 You as Google want to try it, because you know there is a lot of money there. 00:02:21.990 --> 00:02:24.840 But you are afraid of changing something that works pretty well already. 00:02:24.840 --> 00:02:28.710 So what you can do is to pick a random 10% of the users 00:02:28.710 --> 00:02:34.750 of a random small country, say, Belgium, and show to them that new green shape. 00:02:35.810 --> 00:02:37.840 And then you can compare those users to 00:02:37.840 --> 00:02:40.110 other users, where the experiment was not performed. 00:02:41.140 --> 00:02:42.830 Maybe you can also pick another country. 00:02:42.830 --> 00:02:48.370 Say, the Netherlands, and do the same experiment with another color, say orange. 00:02:48.370 --> 00:02:51.466 Or you can try changing the size of the ads, or all at 00:02:51.466 --> 00:02:53.770 the same time, fitting each user into 00:02:53.770 --> 00:02:57.282 their own highly individualized blend of experiments. 00:02:57.282 --> 00:03:00.690 As long as you can extract meaningful data, why wouldn't you do it? 00:03:01.840 --> 00:03:03.230 Well that's what Google does. 00:03:03.230 --> 00:03:07.720 Every time you go on Google hundreds of experiments are actually performed on you. 00:03:07.720 --> 00:03:11.014 The page you get for a particular search, is slightly different 00:03:11.014 --> 00:03:14.890 from the page offered to anyone else for the same search. 00:03:14.890 --> 00:03:16.246 And they try to optimize that page for you. 00:03:16.246 --> 00:03:22.160 This is called A/B testing, and it's used extensively to optimize the web. 00:03:23.280 --> 00:03:24.960 I know it sounds silly to talk about this 00:03:24.960 --> 00:03:27.280 for education but I don't think it really is silly. 00:03:28.740 --> 00:03:32.590 Many blackboards are dyed green because some tests were performed a long 00:03:32.590 --> 00:03:35.510 time ago and people figure out this was better for the eyes. 00:03:35.510 --> 00:03:40.000 You can see the similarity with Google's experiment there. 00:03:40.000 --> 00:03:41.650 Many more studies have been done since 00:03:41.650 --> 00:03:44.380 in education to try and improve other factors. 00:03:44.380 --> 00:03:46.600 My be more meaningful than just a blackboard color. 00:03:46.600 --> 00:03:49.890 But this is slow and requires lots of manual work. 00:03:51.010 --> 00:03:54.200 Again, similar studies can be done in a MOOC space, but 00:03:54.200 --> 00:03:57.390 actually much more is possible, much faster, and much more efficiently. 00:03:59.020 --> 00:04:01.060 For instance, you can alter the material 00:04:01.060 --> 00:04:03.590 presented to the students in very subtle ways. 00:04:03.590 --> 00:04:05.330 And see what has an effect and what does not. 00:04:06.560 --> 00:04:08.810 So let's say you split your class in 00:04:08.810 --> 00:04:11.960 two, and measured how effective it actually is to 00:04:11.960 --> 00:04:17.180 start a class with a little preview of the problem that you want to solve at the end. 00:04:17.180 --> 00:04:21.370 This could be effortless by the teacher, or someone doing educational research. 00:04:22.930 --> 00:04:26.390 The technology to perform these studies exists, and is well understood. 00:04:26.390 --> 00:04:28.690 Because it was used extensively in other industries 00:04:28.690 --> 00:04:31.180 before, and is now being applied to education. 00:04:32.420 --> 00:04:34.870 Some conferences are starting on these topics and releasing 00:04:34.870 --> 00:04:38.590 lots of insight into what works and what does not. 00:04:38.590 --> 00:04:43.560 It's hard to argue when someone comes with a study conducted on millions of students. 00:04:43.560 --> 00:04:48.890 You ask for a follow up and six months later, they come back with that follow up. 00:04:48.890 --> 00:04:52.020 This is tremendously accelerating for education, and very exciting.