In 2008, Australia’s six states and two territories agreed to develop common literacy and numeracy assessments aligned with a national curriculum in these subjects. Top of Class reached out to Barry McGaw, Chair of the Australian Curriculum, Assessment and Reporting Authority (ACARA) and a Professorial Fellow at the University of Melbourne, to discuss the challenges in the development of the national curriculum and assessment system, and lessons for other countries.
NCEE: Can you give us an overview of the history of Australia’s National Assessment and curriculum efforts and what spurred on their development and the decision to create ACARA?
Barry McGaw:Australia has been involved in international studies of student performance since they began in the 1950s; however, the first national surveys of literacy and numeracy occurred in 1976. The evaluations were sample-based only, a bit like NAEP in the United States. And then various Australian states introduced sample-based surveys in other curriculum areas.
Barry McGaw
The first assessment of all students across the country in literacy and numeracy occurred in 1990 in New South Wales, which is the largest state in Australia. It was very controversial at the time; particularly with the teachers unions. So, New South Wales began testing all of their students in literacy and numeracy, and then Victoria, the second-largest Australian state, followed, and then other states gradually joined in. Western Australia participated with a really creative set of sample-based surveys that covered their entire curriculum, but only moved to test the entire student cohort when the then-federal Minister of Education made it a condition of federal funding.
By the mid-2000s, the state Ministers of Education decided it would be good to get results expressed in a way that made them comparable across the country. They instituted a process of creating a common scale allowing each state to continue administering its own assessments, but calibrated onto a common scale. After a couple of years, there was some concern about how good the calibration was, and they said, if we’re all testing in order to get the results on a common scale, why don’t we all use the same tests? And so from 2008, the six states and two territories in Australia have all used the same literacy and numeracy tests nationwide, also known as the National Assessment Program – Literacy and Numeracy (NAPLAN).
The next major event occurred in 2007. During the federal election campaign, the opposition announced that it supported the establishment of a national curriculum in English, Mathematics, Science and History. They won the election at the end of 2007. At the beginning of 2008, they set up an interim national curriculum board. I was appointed chair, with the goal of developing national curriculum in English, Science, Mathematics and History. The government added geography and languages other than English to the curriculum development plans in mid-2008, and then the arts were added later.
At this point, the intention of the federal government was to discuss with the state governments what kind of governance arrangement would be instituted in the longer term for this body. There was some debate about whether it would be set up as a not-for-profit that the ministers collectively owned or a more formal statutory authority of the Australian parliament. The federal minister won out, suggesting that the Council of Ministers (the federal minister and the six state and two territory ministers) would be the policy board for this new body. In late 2008, Australia established the new Australian Curriculum, Assessment and Reporting Authority (ACARA) with responsibility not only for national curriculum development, the national literacy and numeracy assessments, but also the sample-based assessments already underway on a three-yearly cycle for science, civics and citizenship and ICT literacy. Finally the task of public reporting on schools was added to the portfolio. That work has resulted in the creation of the My School website.
NCEE: You mentioned that these national assessments were controversial with teachers unions. Can you talk a little bit about what their concerns were, and how ACARA was able to address them?
Barry McGaw: The teachers unions were unhappy about the public nature of the reporting and this kind of external assessment. They typically argued that these types of assessments don’t tell us anything we don’t already know about our students. The response was, “you know how your students are doing in relation to one another, but you don’t know how your students stand compared to students across the country.” What typically wins the day in this debate is parents. Parents value information that shows them the bigger picture. When the My School website came along, suddenly parents are not just getting reports on their own children, but they’re able to see, collectively, for all the students in their school how they’re doing in comparison with other schools. This information can be controversial, but to help on this front, we include information about the circumstances of the school.
Some other countries do what they call value-added analysis. Here is our approach. We obtain measures of students’ background, in particular on their parents’ education and their parents’ occupation. We’re not trying to measure socioeconomic status, we’re trying to measure socio-educational status – what are the benefits that kids get from the occupation and education of their parents as they come to school. We use this information to create an Index of Community Socio-educational Advantage. And then for every school, we look at just 60 schools – 30 schools immediately above it and the 30 immediately below it on that index. These are schools that have got essentially the same kinds of kids. These are the schools that you can learn from, that can challenge you. Now we can show a school another school with kids exactly like theirs that is doing much better. That school can then ask themselves, what policies they might consider that are different from the ones currently being used? What practices could the school adopt that might reproduce the kind of superior performance that you see in the higher performing schools? So that’s essentially the strategy of the My School website. Not only can it assist parents in their choice of schools, but it underpins attempts at school improvement.
NCEE: How are schools and teachers using the data to improve performance?
Barry McGaw: Well, it’s only been produced twice, so we know that within some of the state systems, they’re using it to help schools make these comparisons, but we don’t have much data ourselves on it yet. We know that huge numbers of people look at the site. States do bring together small groups of schools to look at the data and analyze it. Every school in the country, or ten thousand schools, is in the My School database.
My School website
In terms of test results, we make sure that we only report results for students who were in any given school for each administration of a test for each year it was given. We drop any grade three kids that have gone somewhere else, and we won’t count any grade five kids that have joined you since then, and we’ll compare you with the students in your comparison group of schools with a similar social background, but only in those cases where all of those other schools also have students who were in each school on both occasions. This year, we’re going to provide growth comparisons, which are very interesting and useful, because it begins to give you even further information on what value the schools are adding.
NCEE: Do you plan eventually to use the NAPLAN results to evaluate individual teacher performance in addition to school performance?
Barry McGaw: No. NAPLAN assessments are given only in Grades 3, 5, 7 and 9. It would be very difficult to allocate responsibility for students’ performances or improvements to individual teachers quite apart from the question of how those teaching in grades not tested would be evaluated.
NCEE: Do you see teacher performance as ever becoming a component of the data on the My School website?
Barry McGaw: There are serious discussions going on about how to recognize and reward high-performing teachers but no consideration is being given to reporting on teachers on the My School site.
NCEE: The OECD has recently published a report on evaluation and assessment in Australia (available as a free download from OECD) as part of their international study on these issues. A team of experts visited Australia and observed your system; they made a number of policy observations and suggestions. What did you think about the recommendations that they made for Australia’s system, and are there any that you think the government definitely should implement as you move forward with developing the program?
Barry McGaw: I think it’s a good report. The big thing that we are doing now, as the report pointed out, is developing a strategy for formative assessment. But let me explain where we are first.
The final version of the national curriculum in English, Math, Science and History for kindergarten to grade 10 was adopted last Friday (October 14, 2011), and is now up on the website. It’s quite a historic moment, actually. Already the curriculum is being implemented in the Australian Capital Territory, which is like Washington, DC, because they agreed to the content a year ago. Queensland and South Australia and the Northern Territory will implement next year beginning in January – our school year is the calendar year – and Victoria will have a major pilot in a couple of hundred schools; New South Wales and Western Australia will start in 2013.
What we now have to clarify is the achievement standards. For example, the curriculum states, that, in grade five, in mathematics, these are the things students should have an opportunity to learn. We see our curriculum as a kind of statement of student entitlement. What they should have an opportunity to learn is knowledge, understanding and skills, not just factual stuff. Then we declare in the achievement standards, if a student has satisfactorily learned this, what will a student be able to do? Those statements can be difficult to interpret in any kind of precise way, so what we are doing now, is putting on the website actual samples of students’ work, produced in response to real classroom tasks with annotations to say this student work meets the standards and why. What we will have up by the end of this year, that is by December 2011, for every achievement standard, is some samples of student work. But then next year, while the curriculum is actually being implemented, we’ll be obtaining a richer set of samples illustrating different levels of achievement at the A, B, C, D and E levels. The samples of student work will be annotated, for the first time, by teachers across the country, so that we’ll have nationally annotated samples of student work that can move in the direction of getting consistent use of formative assessments across the country.
NCEE: Will that all be available online?
Barry McGaw: Yes, by the end of next year. And the federal government has just put up funds as well to produce some online assessment resources for teachers.
NCEE: Can you expand on why it’s important to have examples of student work when presenting the new curriculum to educators?
Barry McGaw: You will see on the website, that there are statements of achievement standards to give teachers an idea of what students can do, given the opportunity to develop the knowledge, understanding and skills, set out in a particular part of the curriculum. We think that it is difficult to write such statements in a way that is unambiguous for teachers and that it is much more helpful to also provide samples of real student work in response to real tasks created by teachers, but then assessed by a group of teachers from across the country and annotated to provide an explanation for the judgments they make.
Under the previous federal government there was a requirement introduced that all schools report student performance to parents on an A-E (or equivalent) scale. Our annotated samples of students’ work will illustrate performance for each score, A to E, for each subject, each year. We have collected quite a few this year from schools involved in piloting the K-10 English, Mathematics, Science and History curricula, but will collect more during 2012 as some of the states will have already begun full implementation.
The Council of Education Ministers recently approved the K-10 curricula for English, Mathematics, Science and History on October 14th. You can see details of the implementation plans on our website here.
NCEE: You mentioned the necessity of aligning achievement standards with the national curriculum moving forward. Can you clarify where Australia stands with regard to the link between achievement standards and curriculum content? Were national achievement standards developed before the national curriculum?
Barry McGaw: We think that the curriculum should come first as the expression of the goals of education in terms of the learning entitlements of students. Assessment should follow, shaped by the expectations of student learning.
I recognize that there is a place for ‘assessment-led reform’ where the availability of new forms of assessment can show teachers how to assess learnings that are important but to which they might not attach sufficient significance if they cannot see how to assess them. In that case, it is still the curriculum and its expectations that come first.
NCEE: While building NAPLAN and the national curriculum, what lessons did you draw from other countries? Are there any countries in particular that you used as a model, and in what ways? What do you see as distinctive about the Australian system?
Barry McGaw: NAPLAN grew out of state-based assessments of literacy and numeracy that began in New South Wales in its then Basic Skills Testing Program in 1990. The other states followed over the years. While I was in Paris at the OECD, the Ministers for Education decided that the results should all be expressed on a common scale across the country. The separate tests were equated to achieve this, but then the Ministers decided that it would be better to use common tests. NAPLAN was the result and the first NAPLAN tests were introduced in 2008. Interestingly, there was no common curriculum behind NAPLAN. The new test reflected the separate tests that it replaced.
As part of the development of the Australian Curriculum, ACARA was directed also to develop literacy and numeracy continua and then to review and revise NAPLAN as necessary to reflect those continua. We will time this change on the basis of implementation of the new curriculum with a revised NAPLAN probably to come in 2014.
In our curriculum, we paid attention to practices elsewhere. Our mathematics curriculum, for example, has been increased in difficulty particularly at the elementary school level on the basis of our analysis of mathematics curricula in Singapore and Finland, two countries that outperform Australia in the international comparisons offered by programs such as OECD’s PISA.
NCEE: We know that the NAPLAN assessments are a combination of multiple choice and short answer questions, and are scored electronically and by trained, independent markers. How did you arrive at this system – why are they structured in this way?
Barry McGaw: The form of the test was established before responsibility for it was passed to ACARA. There is a preference in Australia for constructed response questions balanced by cost considerations in favour of machine scoreable responses. As in PISA, the final choice is based on the two considerations.
NCEE: Many people these days think that it is important to measure creativity and the capacity for innovation. Do you agree? If so, how does NAPLAN (or the other sample tests) measure these things? Are these considerations reflected in the national curriculum?
Barry McGaw: They are important but NAPLAN does not measure them. They are in our curriculum, embedded in the subject content not as add-on equivalents to additional subjects. If a teacher wants to focus on creativity, for example, the teacher can apply a filter to the curriculum that will highlight the opportunities that the curriculum in each subject for the school grades of interest to the teacher provides for a focus on creativity. The teacher could use this, for example, in developing an integrating theme through which all the relevant subjects are drawn on. Such a theme could be followed for some days or weeks.
The opportunities will be expanded as we add additional subjects to the Australian Curriculum. Development is now well advanced for Geography and the Arts and the rest are following.
NCEE: What are the lessons that other countries trying to build a national assessment system can draw from Australia?
Barry McGaw: First and foremost is to tie assessment to the curriculum. We’ll probably end up making some adjustments to the literacy and numeracy assessments now that the national curriculum has been adopted. What happened historically in Australia was that each of the states developed its own literacy and numeracy assessments, as I said, but did it in relation to their own curriculum. Then they adopted common assessment practices without having adopted a common curriculum. Now we’ve got the common curriculum as well; we just need to make sure that’s aligned, and the developmental sequences are right. One of the big problems is – and I think this is a legitimate criticism of these kinds of assessment programs – that they can narrow the curriculum. Particularly if you make it really high stakes. And you can’t make it any more high stakes than putting it on a public website like the My School site. So you start to worry about people gaming the system, encouraging poor performing students to stay at home on the day of the assessment, those kinds of things.
To deal with this, we publish right alongside the school’s performance the proportion of students that were in school on the day assessments were given. So, if there’s any obvious manipulation, or indeed, even if there’s not manipulation, if there’s a low participation rate, that’s evident. There is also the question of whether the system can be gamed by narrowing the school’s teaching focus to what you think might prepare students for a particular form of test. Our view is that the research shows that coaching for tests is effective if what it’s doing is making sure the students are familiar with the test’s format. But it is also the case that if you want to prepare your kids’ literacy skills, the way to do it is through a rich curriculum. Kids learn language in history. They learn language in social science studies. They learn numeracy skills with data representations in geography and other areas of social science as well as in math. They learn it in science. So the best way to develop literacy and numeracy is to have a full and rich curriculum. In Hong Kong they use different forms with different kids in the same class. That’s where we’re heading.
NCEE: Is there anything else you would like to talk about with regard to the report, or the direction the system is going in?
McGaw: I’d like to say something about the curriculum itself, rather than the assessment system. When we got started, we were calling what we did the development of content standards. I found out from talking to an American journalist that we borrowed that term from you. I also learned that in the United States you couldn’t talk about national or state curriculum, so you used these words. What we are doing now is saying that we are developing curriculum or the learning entitlements. We say to schools that by whatever means you teach, this is the knowledge, understanding and skills that your kids are entitled to have the opportunity to acquire. You’ve got to get around the constitutional arrangements in order to do the right thing. Australia has strong constitutional arrangements that say that education is the responsibility of the states, not the commonwealth, not the federal government. So how did we get there? We got there by making it a collaborative arrangement. All of this is decided not by the federal minister; all of this is decided by the six states, two territories and the one federal minister sitting at the table together.
Recent Reports of Note
“Quality Counts 2012: The Global Challenge—Education in a Competitive World,” Editorial Projects in Education, Jan. 12, 2012
This report takes a critical look at the nation’s place among the world’s public education systems, with an eye toward providing policymakers with perspective on the extent to which high-profile international assessments can provide valid comparisons and lessons. It examines effective reform strategies in the US and abroad that have gained traction and may be replicable. And, the report highlights the political and social challenges policymakers will face in improving American education to meet the demands of a 21st-century work force. Learn more here.
Andreas Schleicher, “Chinese Lessons,” OECD Education Today Blog, Oct. 14, 2011
Andreas Schleicher, Special Advisor on Education Policy to the OECD’s Secretary General and Head of the Indicators and Analysis Division in the Directorate of Education, recently visited China to launch the OECD’s first-ever Chinese edition of Education at a Glance. He blogs about his visit to an experimental school in Shanghai, China’s particularly successful educational Petri dish where potential nationwide reforms are developed and piloted. Read the full blog post here.
Miller, David C. and Laura K. Warren, “Comparative Indicators of Education in the United States and Other G-8 Countries: 2011,” NCES, October, 2011
Every two years, the National Center for Education Statistics (NCES) releases a compendium of statistics intended to enable comparison between the US and the seven other G-8 countries: Canada, France, Germany, Italy, Japan, the Russian Federation and the United Kingdom. This report focuses on five topical areas – population and school enrollment, academic performance, contexts for learning, expenditures for education and educational attainment and learning. The statistics are drawn from the OECD’s Programme for International Student Assessment (PISA) and their Indicators of Education Systems (INES). To read the full report, click here.
“Education Indicators in Canada: An International Perspective,” Tourism and the Centre for Education Statistics Division, Sept. 2011
This report is intended to be read alongside the OECD’s Education at a Glance 2011 as an in-depth look at the state of Canadian education. Readers interested in Canada’s education system should note the report’s amendments to OECD data; the report points out, for example, that although the OECD statistics show a smaller gap between teachers’ starting and top of scale salaries in Canada, Canadian teachers actually reach the top of the pay scale in half the time of other OECD countries, suggesting a different interpretation of the OECD data. Another notable statistic is the small correlation between students’ reading performance and socioeconomic status; this correlation is far below the OECD average, perhaps indicating particularly successful management of student class disparity in Canada. To read the full report, click here.
“Thematic Probe: Curriculum specification in seven countries,” International Review of Curriculum and Assessment Frameworks, April 2011
INCA’s Thematic Probe provides curriculum and standards information for Australia, Canada, Finland, New Zealand, Scotland, Singapore and South Africa. The information is organized around several questions, as follows: How is the curriculum specified? Are there national standards/expected outcomes? Are curriculum and standards specified and articulated separately or together? Who is responsible for specifying the curriculum? Who is responsible for specifying the standards? How is the curriculum published? Are curriculum components specified locally or nationally? Linked statutory testing – what, when, why? The responses are organized into tables, and provide insight into the links between government control, curricula and standards in some of the top performing countries. To read the full report, click here.
