06 Educational Analytics [Massive Teaching]

Original description:

Sample video for the course #MassiveTeaching on Coursera
Starts June 23rd 2014

https://www.coursera.org/course/massiveteaching

-------
Note:
From Week 1 Lecture Videos of "Teaching goes massive: new skills required"
by Paul-Olivier Dehaye
See
https://etherpad.mozilla.org/pr8ZtLXODg
and
http://www.elearnspace.org/blog/2014/07/09/congrats-to-paul-olivier-dehaye-massiveteaching/

I've talked about one very impressive
study by John Hattie.

He collected data for years where each of
his data points

was itself the result of another study
that took a long time.

That's a lot of work.

I want to explain to you how MOOCs offer
the

opportunity to do all these studies much
more quickly and efficiently.

The first thing to know is every single
click on the major platforms is tracked.

If you were to pause the video now, they
would know.

If you are watching this accelerated, they
know it as well.

They also know how often a student posts
on the

forum or how many time they try to post
questions.

All these days I analysed, some of the
smartest brain

in the world, actually try to make sense
out of it.

The question, the big question is, what
works best for teaching online?

And maybe even what works best for
teaching?

A video of a talking head, beamer slides
with

an insert of the instructor, or maybe
without the instructor.

Some guy's paper cutouts.

Who knows?

This data is analysed, and it's not a
sample size of n equals 20.

But, it's more like n equals two million.

I'm tried to give you a flavor of what is
already possible.

Let's start with a concrete situation.

Let's say you have an ad for a cool
project to change the world.

You want people to help you.

So you write a flyer that say, what to
change the world.

Come and join me at this day and place.

You know that that's not enough to attract
people.

So you want to had, to add a hook before.

You don't want, you don't know for sure
what to put in.

You think maybe, the line, want cookies,
or, want pizza, might attract people.

But what will grab people's attention and
attract people to your purchase/?

You don't know, And, in the real world,
you wont know.

Unless you try both in parallel, and

see afterwards which one is most suc-,
successful.

How do you do that?

Well, by interviewing the people who show
up.

This takes time ,and again, it doesn't
scale well.

Let's say now your Google, you'll
literally

make millions of dollars of revenue per
hour.

Most of it through people clicking on ads
in the sidebar.

One of your employees thinks that a

particular shade of green will definitely
help

highlight the ads, but not too much so
that it doesn't bother the users.

You as Google want to try it, because you
know there is a lot of money there.

But you are afraid of changing something
that works pretty well already.

So what you can do is to pick a random 10%
of the users

of a random small country, say, Belgium,
and show to them that new green shape.

And then you can compare those users to

other users, where the experiment was not
performed.

Maybe you can also pick another country.

Say, the Netherlands, and do the same
experiment with another color, say orange.

Or you can try changing the size of the
ads, or all at

the same time, fitting each user into

their own highly individualized blend of
experiments.

As long as you can extract meaningful
data, why wouldn't you do it?

Well that's what Google does.

Every time you go on Google hundreds of
experiments are actually performed on you.

The page you get for a particular search,
is slightly different

from the page offered to anyone else for
the same search.

And they try to optimize that page for
you.

This is called A/B testing, and it's used
extensively to optimize the web.

I know it sounds silly to talk about this

for education but I don't think it really
is silly.

Many blackboards are dyed green because
some tests were performed a long

time ago and people figure out this was
better for the eyes.

You can see the similarity with Google's
experiment there.

Many more studies have been done since

in education to try and improve other
factors.

My be more meaningful than just a
blackboard color.

But this is slow and requires lots of
manual work.

Again, similar studies can be done in a
MOOC space, but

actually much more is possible, much
faster, and much more efficiently.

For instance, you can alter the material

presented to the students in very subtle
ways.

And see what has an effect and what does
not.

So let's say you split your class in

two, and measured how effective it
actually is to

start a class with a little preview of the
problem that you want to solve at the end.

This could be effortless by the teacher,
or someone doing educational research.

The technology to perform these studies
exists, and is well understood.

Because it was used extensively in other
industries

before, and is now being applied to
education.

Some conferences are starting on these
topics and releasing

lots of insight into what works and what
does not.

It's hard to argue when someone comes with
a study conducted on millions of students.

You ask for a follow up and six months
later, they come back with that follow up.

This is tremendously accelerating for
education, and very exciting.