PISA - Measuring student success around the world

A test the whole world can take...

It's a breathtaking concept -- but for the OECD, it's a very real way of finding out how well-prepared today's students are to participate in society. If you're not quite sure what PISA is or does, and you've got about 12 minutes to spare, grab a comfortable chair. This video will help you to understand why the OECD's number-crunchers trigger such intense debate about the state of education around the world every time they release the results of the latest PISA survey.

You'll learn a lot—and you won't be tested on any of it!

1
00:00:11,093 --> 00:00:16,044
If you read or listen to news reports about education you've probably noticed periodical surge of interest in wich country students do best in

2
00:00:16,044 --> 00:00:21,090
readings or mathematics or science and where your country fits into the grand scheme of things

3
00:00:22,001 --> 00:00:28,083
you have probably also heard, or read the word PISA, in connection with some of these reports. What

4
00:00:28,086 --> 00:00:34,098
is PISA? PISA is an achronym that stands for the Programme for International Student Assessment. It's

5
00:00:34,098 --> 00:00:44,037
the brain child of the OECD, and what's the OECD? It's another achronym that stands for the Organization

6
00:00:44,037 --> 00:00:51,065
for the economic cooperation and development. The OECD brings toghether 34 countries, with the aim of developing

7
00:00:51,065 --> 00:01:00,034
better policies for better lives. In the late 1990's countries that are members of the OECD came up with

8
00:01:00,034 --> 00:01:07,030
the idea to measure whether fifteen years olds around the world are well prepared to participate in society.

9
00:01:07,034 --> 00:01:14,030
We choose fifteen years olds rather than twelve or seventeen year olds because most fifteen years old are about

10
00:01:14,030 --> 00:01:21,059
to complete their compulsory education. Experts in the fields of education from around the world worked

11
00:01:21,059 --> 00:01:28,067
toghether to create a two hour test, that focuses on core subjects like reading mathematics and science.

12
00:01:28,067 --> 00:01:35,077
Participating countries deciced to administer this test every three years and to rotate the main focus

13
00:01:35,080 --> 00:01:45,030
of the test among the core subjects. All very well, but testing students is nothing new, so what's so

14
00:01:45,030 --> 00:01:51,032
special about PISA? PISA surveys are design to find out whethet students can use what they have learned

15
00:01:51,032 --> 00:01:58,058
in school, and apply that knowledge to real live situations and problems. PISA is less interested in

16
00:01:58,058 --> 00:02:04,080
knowing whether students can repeat like parrots what they have been thaught in class. Rather the survey

17
00:02:04,080 --> 00:02:11,019
is designed to find out whether, for example, students can use their reading skills they have learned

18
00:02:11,019 --> 00:02:17,036
in schools, to make sense of the information they find in a book, a newspaper, on a government form,

19
00:02:17,036 --> 00:02:23,049
or in an instruction manual, but the point of PISA is not to tell each individual student how well he

20
00:02:23,049 --> 00:02:32,009
or she has mastered the set of skills. Instead, PISA results are extrapolated and analysed to the national

21
00:02:32,009 --> 00:02:40,052
level. Picture one student sitting at a desk, in a classroom somewhere, taking the PISa test, now zoom

22
00:02:40,052 --> 00:02:45,053
out as though you are on a space shuttle, and you can see the entire country in wich that student is

23
00:02:45,053 --> 00:02:53,098
sitting. That's what PISA does with its test results, PISA shows countries where they stand, in relation

24
00:02:53,098 --> 00:03:00,095
to other countries, and just by themself, in how effectively they educate their children. What PISA doesn't

25
00:03:00,095 --> 00:03:08,061
say: this education policy or practice causes that effect, it shows what's possible, and show similarities

26
00:03:08,061 --> 00:03:15,039
and differences between education systems around the world. That helps governments rethink their own

27
00:03:15,039 --> 00:03:22,022
policies and design new ones to improve their student performance in school. It also helps government

28
00:03:22,022 --> 00:03:30,041
educators, and parents track their country's progress towards a more succesful education system. In fact

29
00:03:30,041 --> 00:03:39,066
many countries now set national goals, and benchmarks based on PISA international results. Pisa considers

30
00:03:39,066 --> 00:03:45,058
and education system succesful not only if its students achieve high scores on the PISA surveys, but

31
00:03:45,058 --> 00:03:53,012
also if all students, from all backgrounds perform well on the test. Not just those who come from

32
00:03:53,012 --> 00:04:00,014
wealthier, or more intellectual, or more culturally sophisticated families. For example a relatively

33
00:04:00,014 --> 00:04:08,026
large percentage of disadvantaged students in places like Hong Kong, Shanghai, Korea, and Finland achieve

34
00:04:08,026 --> 00:04:15,079
some of the highest scores in PISA. Analysts there look at the PISA test results along with responses

35
00:04:15,079 --> 00:04:21,008
to questionnaires that are given to students and school principals, and try to determine the main characteristics

36
00:04:21,008 --> 00:04:28,056
of these succesful education systems. Are teachers in the system paid more? Are classes generally larger

37
00:04:28,056 --> 00:04:36,006
or smaller? Do individual schools get to decide what their teachers teach? Or is their curriculum determined

38
00:04:36,006 --> 00:04:43,074
by a central government authority? Once a profile of a of a succesful system emerges, it can be used

39
00:04:43,074 --> 00:04:52,015
as a model for others. So what's the test like? Here are a couple of problems that students have to solve.

40
00:04:52,015 --> 00:04:58,053
The first one was part of the reading assessment. Students were shown a graph and an accompanying text.

41
00:04:58,053 --> 00:05:12,058
that read: figure one shows changing levels of Lake Chad, in Saharan Nord Africa. Lake Chad disappeared completely in about 20000 B.C. during the last ice age. In about 11000 B.C. it

42
00:05:12,058 --> 00:05:20,059
reappeared. Today its level is about the same as it was in A.D. 1000. Students whare asked why has the

43
00:05:20,059 --> 00:05:27,058
author chosen to start the graph at this point. This was considered a rather difficult question. In fact,

44
00:05:27,058 --> 00:05:35,085
across OECD countries, only 37% of students answered it correctly. Those students showed ana ability

45
00:05:35,085 --> 00:05:42,023
not only to read, but to think about what they read, and draw some conclusion from it. They figured out,

46
00:05:42,023 --> 00:05:48,092
that the reason the graph starts at around 11000 B.C. is because that's when Lake Chad reappeard after

47
00:05:48,092 --> 00:05:56,049
having disappeared completely, during the ice age. The second problem was part of the mathematics assessment,

48
00:05:56,049 --> 00:06:04,055
it sets the scene. In a pizza restaurant you can get a basic pizza, with two toppings: cheese and tomato.

49
00:06:04,055 --> 00:06:12,097
You can also make up your own pizza with extra toppings, you can choose from four different extra toppings.

50
00:06:12,097 --> 00:06:21,076
Olives, ham ,mushrooms and salami. Ross wants to order a pizza with two different extra toppings. Students

51
00:06:21,076 --> 00:06:28,049
where then asked: how many different combination can Ross choose from? Now this is a problem we can all

52
00:06:28,049 --> 00:06:34,046
relate to. To answer the question students had to show their ability to make connections. In this case

53
00:06:34,046 --> 00:06:41,003
juggling several food stuff and realize that the choices are not, in fact, infinite, much as the students

54
00:06:41,003 --> 00:06:48,092
might like them to be. This was also a relatively tought problem for students to solve. Just 49% of students

55
00:06:48,092 --> 00:06:56,072
could calculate that only six different variation where possible. So when we take individuals score in

56
00:06:56,072 --> 00:07:03,024
PISA and the responses to those questionnaires that we circulate with the test and then zoom out to see

57
00:07:03,024 --> 00:07:09,080
a whole country, what kinds of things do we learn. Well, one thing we've learned is girls do better in reading

58
00:07:09,080 --> 00:07:16,030
than boys in every country that participates in PISA, and among the countries that are members of the OECD

59
00:07:16,030 --> 00:07:22,073
girls do so much better than boys in reading, it's as if they have gone through an additional year of

60
00:07:22,073 --> 00:07:30,063
school. We also find out that boys generally do better than girls in mathematics, and that there is no

61
00:07:30,063 --> 00:07:36,092
real difference between boys and girls in how they do in science. We also found out that some school

62
00:07:36,092 --> 00:07:44,067
policies may not be very good for students. For example, early tracking of students, wich means deciding that

63
00:07:44,067 --> 00:07:49,099
some students should go through an academic program, while others should go through a vocational program

64
00:07:49,099 --> 00:07:56,070
is not associated with better overall performance, tracking is also related to greater inequalities

65
00:07:56,070 --> 00:08:03,085
between advantaged and disadvantaged students. Students are often tracked in the mistaken believe that

66
00:08:03,085 --> 00:08:09,080
not everyone can learn the same things, that only some children are gifted and reach for the stars, but

67
00:08:09,080 --> 00:08:18,013
PISA results show that if given the opportunity and support to excel, all children have the potential

68
00:08:18,013 --> 00:08:26,071
to do so. Having students repeat grades is also not associated with high scores in PISA. School system

69
00:08:26,071 --> 00:08:32,070
that invest in helping strudents learn their subjects the first time around do much better than those

70
00:08:32,070 --> 00:08:39,068
where teachers know that they can ,if necessary drill the same material, year after year, after year

71
00:08:39,069 --> 00:08:46,050
into the heads of the same struggling students. As we mentioned before the most successful schools according to

72
00:08:46,050 --> 00:08:53,012
PISA are the ones whose students do well regardless of where they come from. Still results from PISA

73
00:08:53,012 --> 00:09:01,021
show that home background has a major influence on student success in school. In many ways this finding

74
00:09:01,021 --> 00:09:06,078
is all too obvious. We know for example that by the time they are three, children in advantaged families

75
00:09:06,078 --> 00:09:15,034
are exposed to many more word than their less advantaged peers. In fact, a recent study in USA put their

76
00:09:15,035 --> 00:09:21,048
number at around thirty million more words, and in general if there are no books at home, or if their

77
00:09:21,048 --> 00:09:28,058
children don't see their parents reading they'll be less inclined to read themself. Pisa results alo

78
00:09:28,058 --> 00:09:35,059
show that regardless of their own background, student who attend schools that have a largely disadvantaged

79
00:09:35,059 --> 00:09:43,075
population tend to do worse than students who attend schools with relatively advantaged peers. Why would

80
00:09:43,075 --> 00:09:50,039
this be so? Well, there are many possible reasons, for example PISA found that in most OECD countries

81
00:09:50,039 --> 00:09:56,038
disadvantaged students have access to the same number of teachers and to sometimes even more teachers than their more advantaged peers.

82
00:09:56,038 --> 00:10:08,052
The problem though is that more isn't necessarily better. In fact, the best teachers are often found in

83
00:10:08,052 --> 00:10:15,089
schools attended by advantaged students who generally do well on their subjects anyway, but not in disadvataged

84
00:10:15,089 --> 00:10:24,034
schools where high quality teachers are most desperately needed. Governments around the world can be inspired

85
00:10:24,034 --> 00:10:29,062
by two significant finding from PISA. The first is that a country doesn't have to be wealthy to provide

86
00:10:29,062 --> 00:10:36,087
high quality education to its students. Shanghai and Poland , for example, score well above the OECD average inreading,

87
00:10:36,087 --> 00:10:44,098
but rank below OECD average in measures of national wealth, and the countries PISA ranking is not carved in

88
00:10:44,098 --> 00:10:52,092
stone. Trends in PISA have revealed a great capacity for all countries to improve. Countries as diverse

89
00:10:52,092 --> 00:11:00,010
as Chile, Germany, Poland and Portugal among others, showed improvements in student reading performance

90
00:11:00,010 --> 00:11:08,006
between 2000 and 2009, although it seems obvious, it's still worth reminding ourselves. Successful education

91
00:11:08,006 --> 00:11:15,037
systems make education a priority, they share the belief that skills can be learned and that all students

92
00:11:15,037 --> 00:11:21,045
can achieve at high levels. They show that they value the teacheing profession by investing in it, so

93
00:11:21,045 --> 00:11:29,012
that they can attract highly qualified candidates, train them well and retain the best teachers among

94
00:11:29,012 --> 00:11:34,020
them. Just as every student has the potential to achieve, every country has the potential to raise the

95
00:11:34,020 --> 00:11:42,089
standards of their education, so PISA is about a lot more than test scores. Sure countries that participate

96
00:11:42,089 --> 00:11:49,023
in the PISA survey are keen to know where their students rank compared to students in other countries.

97
00:11:49,023 --> 00:11:57,003
but PISA's ultimate aim is not to create a competition for its own sake, its aim it's to encourage

98
00:11:57,004 --> 00:12:03,011
all participating countries to use the survey findings to improve their own teaching and student performance.

99
00:12:03,011 --> 99:59:59,000
To give their students the best opportunities to achieve the best possible results.