Anti ed reformers make the case for PISA

In this article from the tireless Valerie Strauss of the Washington Post, Pasi Sahlberg and Andy Hargreaves respond to the open letter signed by dozens of researchers and academics from around the world to Andreas Schleicher, director of the Program of International Student Assessment, urging him to suspend administration of PISA until a new exam can be created.

The Sahlberg and Hargreaves arguments rests on the following:

1. Ignorance is not great and could have given corporate ed reformers an even easier ride

Just think for a moment what would global education look like if PISA had never been launched? There would be, as there was in the 1990s, a number of countries that mistakenly believed their education systems are the best in the world and should set the direction for other nations. Were it not for the fact that these weaker performing countries that include the United States and England have not been successful in PISA, the worldwide pressures for more market competition between schools, less university-based training for teachers, and more standardization of the curriculum, would have had a far easier ride.

The poor performance of Sweden after the implementation of their radical market choice program of for profit free schools would never have been outed.  Likewise the high performance of public education focussed Finland and to a lesser extent Canada would not have provided a very strong counter narrative.

2. PISA has enabled the OECD to shine a bright light on equity and to argue that equity and quality are not at odds:

It has put equity high up on the reform agenda. Without the data that PISA has generated over the years, calls for enhanced equity would not be part of the education policy conversation in the countries that have suffered from inequitable education systems, including the U.S. [and Australia].

However the authors do not let PISA off the hook on the many other issues raised by the group of academics and researchers.  In particular they raise serious concerns about the recent steps to put the tests into the hands of global corporate ed reformers, Pearson.

The conclude that the a) evidence provided by PISA is overwhelming and clear on the negatives of neoliberal education policies and b) that the negatives of PISA can be addressed by dealing with its problems not “knocking the PISA tower over”:

What PISA shows to the United States is that its current course of education policies that rely on competition, standardization, testing and privatization of public education is a wrong way. Our goal should not be to take PISA down, but to get it or something like it upright again, so that by using a range of criteria, and by using them in a fair and transparent way, we can identify and learn from the true high performers who are strong on equity as well as excellence, and on human development as well as tested achievement.

What do readers think?

NAPLAN DAY – What did your child do today: go to the zoo or sit a test?

Today is the start of NAPLAN day for every Australian parent with a child in years 3,5,7 or 9. The vast majority of parents will send their children off to school as per usual, perhaps with an extra hug and an exhortation to” just do your best and don’t get stressed”.

But for a small, but growing, number of parents, this is a day to do something quite different – to go to the movies, the zoo, a picnic or just stay home and have a pajama day. They have taken the decision to remove their child from testing.

Now there are no rights or wrongs about this. It is a personal decision. But you may be wondering why people are making this decision.

I have been reading the many testimonials from US parents about why they have come to this decision and the few statements I have come across about withdrawal decisions from Australian parents. In this piece I bring together the key reasons.

Here is one US parent speaking:

 As a nation we have been convinced that our public schools are failing, that the “status quo” is unacceptable, that schools need standards and testing in order to succeed, and that market based reforms such as privatization, charter schools, vouchers and “dumping the losers” are the way to get it done.  The only problem is that none of this is true. None of it…..

It is the test that binds all of this insanity together.  Without the tests, the reformers have nothing to threaten schools with.  Without the tests, the federal government loses power over states.  Without the tests, schools would be able to stop assigning multiple choice tests to kindergarteners.  Without the tests, there would be no way for education reformers to convince you that your schools are much worse than they really are.  Without the tests, there wouldn’t be a target on our teachers.

But tests aren’t really the problem, the real problem is how the tests are used. Tests are an important form of data that can help educators determine how students are doing and how they need to improve.  When used for that purpose, tests are great.  Still limited, but great.  However, when used as a tool for propaganda, profit and pressure, tests are more punitive than positive.  As long as high stakes standardized tests – despite their limitations – are used as the primary means for evaluating schools, they will continue to be far more valuable for punishing states, schools and teachers than for evaluating student achievement.

There isn’t much I can do about this as an educator and an academic other than write and speak when I’m allowed.  But as a parent I have the power to take control over the education of my child, and that’s exactly what my wife and I have decided to do.


This opt out movement in the US started as a mere trickle but this year it has reached a critical mass. In Long Island alone more than 20,000 school children did not take the first round of state tests that began April 1[1].

Here is another parent – this one not a teacher – explaining her decision to opt out

Lawmakers and education reformers are pushing policies that subtract joy from the classroom, and as a parent of two public school students I am looking to push back. That’s why I joined the opt-out movement ..

Lawmakers and education reformers are pushing policies that subtract joy from the classroom, and as a parent of two public school students I am looking to push back. That’s why I joined the opt-out movement ..

…this year their father and I refused to send our kids to school for …testing. Instead they slept in, watched TV, played outside and read for pleasure. Their grandma also took them to the museum….

I’ve come to believe standardized tests are to learning as an exhibit of butterflies is to nature. In the attempt to pin down what is measurable, we render something wild and beautiful, dead and on display.

While our public school leaders pay lip service to creativity and innovation, they are mandating more class time be devoted to standardized testing in the name of holding teachers accountable for student progress. Next year, Colorado charges headlong into a pay-for-performance system tying 50 percent of our public school teachers’ evaluation to student progress.

Ravitch, … believes parents can halt this parasitic process by refusing to allow students to take the tests that feed it. “Deny them the data,” is the slogan inspiring me and thousands of parents around the country.


But my personal favourite is this letter from Will and Wendy Richardson from Delaware

To the Editor:

After much thought, we have decided to keep our son home during …standardized assessments …. we are basing this decision on our serious concerns about what the test itself is doing to our son’s opportunity to receive a well-rounded, relevant education, and because of the intention of state policy makers to use the test in ways it was never intended to be used. These concerns should be shared by every parent and community member who wants our children to be fully prepared for the much more complex and connected world in which they will live, and by those who care about our ability to flourish as a country moving forward.

Our current school systems and assessments were created for a learning world that is quickly disappearing. In his working life, my son will be expected to solve real world problems, create and share meaningful work with the world, make sense of reams of unedited digital information, and regularly work with others a half a world away using computers and mobile devices. The NJ ASK tells us nothing about his ability or preparedness to do that. The paper and pencil tasks given on the test provide little useful information on what he has learned that goes beyond what we can see for ourselves on a daily basis and what his teachers relay to us through their own assessments in class. We implicitly trust the caring professionals in our son’s classroom to provide this important, timely feedback as opposed to a single data point from one test, data that is reported out six months later without any context for areas where he may need help or remediation. In short, these tests don’t help our son learn, nor do they help his teachers teach him. 

In addition, the test itself poses a number of problems:

         Over the years, the “high stakes” nature of school evaluation has narrowed instruction to focus on only those areas that are tested. This has led to reductions in the arts, languages, physical education and more.

         Research has shown that high scores can be achieved without any real critical thinking or problem solving ability.

         The huge amount of tax dollars that are being spent on creating, delivering and scoring the tests, dollars that are going to businesses with, no surprise, powerful lobbyists in the state capitol and in Washington, DC, is hugely problematic.

         Proposals to use these test scores for up to 50% of a teacher’s evaluation are equally problematic. The tests were not created for such a use, and to create even higher stakes for the NJ ASK will only create more test prep in our classrooms at the expense of the relevant, authentic, real world learning that our students desperately need.

         These tests create unnecessary anxiety and stress in many students who feel immense pressure to do well.

In no way are we taking this step because of our dissatisfaction with our son’s public school, the teachers and administrators there, or our school board. We have simply had enough of national and state policies that we feel are hurting the educational opportunities for all children. At the end of the day, we don’t care what our son scores on a test that doesn’t measure the things we hold most important in his education: the development of his interest in learning, his ability to use the many resources he has at his disposal to direct his own learning, and his ability to work with others to create real world solutions to the problems we face. And we feel our tax dollars are better spent supporting our schools and our teachers who will help him reach those goals as well as the goals detailed by the state standards in ways that are more relevant, engaging and important than four days of testing could ever accomplish.

There are many many parent testimonials to opting out and many impassioned arguments about why they feel it necessary to take this step. But for me the following themes appear to stand out:

  1. The problem isn’t testing per se – but how tests are used –  the lack of validity and reliability in their unintended uses. This testing culture punishes and diminishes teachers.

 In the US this is particularly problematic, because of federal Government mandates that require states to use standardized tests as one of the measures to assess teachers. This was mooted by Ben Jenson from the Grattan Institute at one point and also by Julia Gillard. But because of excellent intervention by AITSL this disastrous situation has been avoided – at least for now.

But we do use NAPLAN scores as the basis for student outcomes reporting on the MySchool website. This turns these tests from a low stakes test to a high stakes event, uses the data in ways that are psychometrically questionable and fosters an unhealthy market choice model of education.

  1. The testing culture has impoverished what happens in classrooms and parents want education to be a joyful experience and to prepare students for active participation as adults in social, economic and political life. The kind of learning that can be tested will not equip students for this.

It is interesting to note that almost none of the testimonies I located were from parent who had children who were stressed or made sick by testing days. This is not to suggest that this situation does not exist , but that this is not what is driving the opt out movement. These are parents who want education to be the best it can be for all students and see the testing culture as undermining that, not just for their child but for all students.

  1. We don’t want to be part of the problem, so we are pushing back, refusing to provide our data to a bad process. In this way we haope to be part of building a movement that will destroy the corporate education stranglehold on our nation’s education.

Many many parents were at pains to state that they don’t believe there is a crisis in public education in the US and that they trust teachers as professionals more than they trust a multiple choice test to assess their childrens’ progress

How will you know what your child is capable of if you don’t have test scores?”  The answer to that is pretty simple.  We trust our son’s teachers.  The privileging of standardized test score data above all other forms of information regarding a student’s progress is a relatively recent phenomenon.  There was a time when we trusted teachers to teach, assess, and evaluate the progress of our students.  We believe this should still be the case.  We don’t need standardized tests to tell us what our kids are capable of.  Our sons’ teachers are more than capable of evaluating and communicating our son’s capabilities in the class using the data they collect through classwork, teacher created assessments and other formative data points that aren’t mandated by the federal government.  Did you know that the new assessments for CCSS will be graded completely by a computer?  Even students’ writing will be scored by a computer.  They’ll tell you that algorithms can be constructed to evaluate a human’s writing capacity.  As an expert in how kids think and learn, I’ll tell you that’s ridiculous.  Testing is one of the least authentic ways to determine  what any child is capable of. Nowhere else in life do we try to determine what somebody is capable of by putting them in front of a test and asking them to fill in bubbles.  Yet in in American public education, that’s quickly becoming the ONLY way we determine what students are capable of.

In Australia one person who has gone public about his decision to withdraw his eldest child from NAPLAN testing is Glen Fowler, ACT branch secretary of the Australian Education Union.

He has withdrawn his year 3 child, because NAPLAN data is published to show how individual schools are performing.

The use of this data to compare and rank schools is a disingenuous practice, and from my point of view, if the data is being misused, there will be no data provided by my family….

I’ve got no issue with standardised tests which are low stakes – I’ve got no issue with sample testing which is done by PISA [Program for International Student Assessment] every year … there’s no capacity for that to damage the reputation of a school or a teacher or a student.

If I had kids of NAPLAN age I would definitely withdraw them, not because of concerns about the effects on my child but as a political act. If enough parents acted in this way, the results would become even more unreliable and eventually there might need to be an acknowledgement that this is not our best policy. NAPLAN is NOT diagnostic; it narrows the curriculum and encourages low-level thinking, and it harming some children[2].

Maybe all this could be seen to be acceptable if there was a more important upside to the enterprise, When the decision to publish NAPLAN results to the school level o MySchool was first announced, there were many noble speeches about using NAPLAN to assess which children and which schools need extra help so that resources can be appropriated for this purpose,  But NAPLAN is NOT being used to identify those schools needing extra funding. And with tonight’s budget decision I very much fear, school funding in Australia will continue to ignore the needs of our most disadvantaged students. In this context NAPLAN is nothing but a cruel joke.


[2] if you want to think through your position on NAPLAN the ‘Say no to NAPLAN’ site established by Literacy Educators at Sydney University provides an excellent set of papers about why NAPLAN is problematic.


The NAPLAN Parliamentary Review’s ‘do nothing’ recommendations: We can do better

Many of us waited with a degree of eagerness – even excitement – for the release of the Parliamentary Inquiry Report into NAPLAN (Effectiveness of the National Assessment Program – Literacy and Numeracy Report). But what a disappointment!

While it makes a passable fist of identifying many, but by no means all, of the significant issues associated with how our NAPLAN is currently administered and reported, it does miss some of the important details. This could be forgiven if the recommendations showed any evidence of careful thinking, vision, or courage. But in my assessment they are trivial and essentially meaningless

We know a lot about the problems with our current approach to standardised testing and reporting. This Report, with the help of over 95 submissions from a wide range of sources, manages to acknowledge many of them. The key problems include:

  • it is not valid and reliable at the school level
  • it is not diagnostic
  • the test results take 5 months to be reported
  • it is totally unsuitable for remote Indigenous students – our most disadvantaged students – because it is not multilevel, in the language that they speak or culturally accessible (Freeman)
  • now that it has become a high stakes test it is having perverse impacts on teaching and learning
  • some of our most important teaching and learning goals are not reducible to multiple choice tests
  • there is a very real danger that it will be used to assess teacher performance – a task it is not at all suited to
  • some students are being harmed by this exercise
  • a few schools are using it to weed out ‘unsuitable enrolments’
  • school comparisons exacerbate the neoliberal choice narrative that has been so destructive to fair funding, desegregated schools and classrooms and equitable education outcomes
  • there will always be a risk of league tables
  • their unequal impact on high needs school
  • they do not feed into base funding formulas. In spite of the rhetoric about equity and funding prioritization being a key driver for NAPLAN, it is not clear that any state uses the NAPLAN to inform their base funding allocations to schools[1]

However, the ‘solutions’ put forward by the report are limited to the following recommendations:

  1. develop on-line testing to improve test results turn around – something that is happening anyway
  2. take into account the needs of students with a disability and English language learners. Now this recommendation is so vague as to be meaningless
  3. have ACARA closely monitor events to ensure league tables are not developed and that the results feed into funding considerations. This is another vague do nothing recommendation and I am certain ACARA will say that they are already doing this.

This is a recommendation to do nothing – nothing that is not already being done or nothing of meaningful substance.

As an example of the paucity of its analysis I offer the following. The report writes about the lack of diagnostic power of the NAPLAN tests and then says that, even if they were diagnostic, the results come too late to be useful. The report then argues, as its first and only strong recommendation that there needs to be quicker timeframe for making the results available. Did the writer even realize that this would still not make the tests useful as a diagnostic tool?

This Report, while noting the many problems assumes that these can be addressed through minor re-emphasis and adjustments – a steady as she goes refresh. However the problems identified in the Report suggest that tiny adjustments won’t address the issues. A paradigm change is required here.

We are so accustomed now to national standardised testing based on multiple choice questions in a narrow band of subjects being ‘the way we do things’, that it seems our deliberations are simply incapable of imagining that there might be a better way.

To illustrate what I mean I would like to take you back to the 1990s in Australia – to the days when NAPLAN was first foisted on a very wary education community.

How many of us can remember the pre national testing days? Just in case I will try and refresh your memory on some key elements and also provide a little of the ‘insider debates’ before we adopted the NAPLAN tests.

1989 was the historic year when all Education Ministers signed up to a shared set of goals under the now defunct 1989 Hobart Declaration. Australia was also in the process of finalising its first ever national curriculum – a set of Profiles and Statements about what all Australian children should learn. This was an extensive process driven by an interstate committee headed by the then Director of School Education in NSW, Dr Ken Boston.

During this time, I worked in the mega agency created by John Dawkins, the Department of Employment, Education, Training and Youth Affairs, initially in the Secretariat for the Education Ministerial Council (then called the AEC) and a few years later heading up the Curriculum and Gender Equity Policy Unit.

The Education Division at that time was heavily engaged in discussion with ACER and OECD about the development of global tests –the outcomes of which are PISA and a whole swag of other tests.

This was also when standardised testing was also being talked about for Australian schools. Professor Cummings reminds us of this early period in her submission to the Parliamentary Inquiry when she says that

This was also when standardised testing was also being talked about for Australian schools. Professor Cummings reminds us of this early period in her submission to the Parliamentary Inquiry when she says that

…through the Hobart Declaration in 1989 … ‘Ministers of Education agreed to a plan to map appropriate knowledge and skills for English literacy. These literacy goals included listening, speaking, reading and writing….

National literacy goals and sub-goals were also developed in the National Literacy (and Numeracy) Plan during the 1990s, including: …comprehensive assessment of all students by teachers as early as possible in the first years of schooling…to ensure that…literacy needs of all students are adequately addressed and to intervene as early as possible to address the needs of those students identified as at risk of not making adequate progress towards the national…literacy goals….use [of] rigorous State-based assessment procedures to assess students against the Year 3 benchmark for…reading, writing and spelling for 1998 onward.

It is interesting to note that the on entry assessments of children by teachers commitment referred to by Cummings did result in some work in each state. But it never received the policy focus, funding or attention that it deserved., which I regard as a pity The rigorous assessments at Year 3 however grew in importance and momentum. But the key consequence of this commitment was the year 3 state based assessment. Professor Cummings goes on to say that in order to achieve this goal – a laudable goal – the NAP was born. State based at first with very strict provisions about not providing school based data and then eventually what we have today.

Professor Cummings may not have known that many of us working in education at the time did not believe that national multiple choice standardised tests were the best and only answer and that considerable work was undertaken to convince the then Education Minister, Kim Beazley, that there was a better way.

During this period, where National standardised literacy tests were being discussed in the media and behind closed doors at the education Ministers’ Council,

Over this same period the US based Coalition of Essential Schools was developing authentic classroom teaching and learning activities that were also powerful diagnostic assessment exercises. Its stated goal was to build a data bank of these authentic assessments activities and to benchmark student progress against these benchmarks across the US. Its long term goal was to make available to schools across the US a data-base of benchmarked (that is standardised) assessments with support materials about how to use the materials as classroom lessons and how to use the results to a) diagnose a students learning b) plan future learning experiences and c) compare their development to a US wide standard of literacy development.

As the manager of the curriculum policy area, I followed these developments with great interest, as did a number of my work colleagues inside and outside the Department. We saw the potential of these assessments to provide a much less controversial, and less damaging way of meeting the Ministers’ need to show leadership in this area.

Our initiatives resulted in DEETYA agreeing to fund a trial to develop similar diagnostic classroom friendly literacy assessment units as the first part of this process. We planned to use these to demonstrate to decision makers that there was a better solution than standardized multiple-choice tests.

As a consequence I commenced working with Geoff Masters (then at ACER as an assessment expert) and Sharon Burrows (who headed up the Australian Education Union at the time) exploring the potential power of well designed formative assessments, based on authentic classroom teaching formats, to identify those at risk of not being successful at developing literacy skills.

Unfortunately we failed to head off a decision to opt for standardised tests. We failed for a number of reasons:

  • the issue moved too quickly,
  • the OECD testing process had created a degree of enthusiasm amongst that data crunchers who had louder voices,
  • our proposal was more difficult to translate to three word slogans or easy jargon,
  • multiple choice tests were cheaper.

At the time I thought these were the most important reasons. But looking back now, I can also see that our alternative proposal never had a chance because it relied on trusting teachers. Teachers had to teach the units and assess the students’ work. What was to stop them cheating and altering the results? Solutions could have been developed, but without the ICT developments we have access to today, they would have been cumbersome.

I often wonder what would have happened if we had initiated this project earlier and been more convincing. Could we have been ‘the Finland’ of education, proving that we can monitor children’s learning progress, identify students at risk early in their school lives, prioritise funding based on need  – all without the distorting effects of NAPLAN and MySchool?

We can’t go back in time but we can advocate for a bigger, bolder approach to addressing the significant problems associated with our current NAPLAN architecture. The parliamentary report failed us here but this should not stop us.

I have written this piece because I wanted us to imagine, for a moment, that it is possible to have more courageous bold and educationally justifiable policy solutions around assessment than what we have now. The pedestrian “rearrange the deck-chairs” of this Report is just not good enough.

So here is my recommendation, and I put it out as a challenge to the many professional education bodies and teacher Education Institutions out there.

Set up a project as follows:

Identify a group of our most inspiring education leaders through a collaborative peer nomination process. Ensure the group includes young and old teachers and principals, teachers with significant experience in our most challenging schools especially our remote Indigenous schools. Provide them with a group of expert critical friends – policy experts, testing and data experts, assessment and literacy experts and ask them to:

  • Imagine there is no current assessment infrastructure
  • Devise an educationally defensible assessment architecture – taking a green fields approach

I can almost guarantee that this working group would not invent NAPLAN and MySchool or anything like it, and we would be significantly better off.

We have dug ourselves into an educationally indefensible policy hole because we have allowed politicians and the media to drive key decision. To my knowledge we have never established an expert group of educational practitioners with access to specialist expertise to develop better policy solutions in education. Why don’t we give it a try?

Any takers?

[1] I understand that NSW does use the NAPLAN results to channel some additional funds to low performing schools but these are above the line payments.

Is opting out of testing just selfish individualism?

In a recent article about American culture and the opt out society Alan Greenblatt described the growing and successful movement to encourage parents to refuse to allow their child to participate in national standardised testing as selfish individualism.  It might be driven by a parents individual interest, he argues, but it is selfish and against collective interests:

 It’s probably true that the time spent on testing isn’t going to be particularly beneficial to the kids, but it’s very beneficial to the system,” says Michael Petrilli, executive vice president of the Fordham Institute, an education think tank. “If you have enough people opt out of these tests, then you have removed some important information that could make our schools better.

I find this amusing because the whole corporate reform movement, for which testing is the centerpiece, is built on the neoliberal belief that the best solution to everything – prisons, health, education etc – is to turn everything into a market and allow competition and individual choice to drive better value.

In fact this was the prime motivation described by Kevin Rudd when he first announced the ‘school transparency agenda’ on the 21 August 2008 at the National Press Club. The speech has mysteriously disappeared but I am quite clear that Kevin Rudd said something along the following lines

“If parents are unhappy with their local school because of the information in MySchool, and decide to transfer their child to another better performing school, then that is exactly what should happen.  This is how schools will improve, through parents voting with their feet.”

Now nobody who works in a struggling school thinks this is the way schools improve. Australia has run an aggressive market choice model of school funding for nearly 2 decades now and all we have to show for it is a highly class segregated schooling system and high levels of inequality.

So let me reassure parents who are concerned about our high stakes NAPLAN testing regime.  Opting out of having your child participate in these tests is much more of a community act than deciding to send your child to an elite school.

Holding Ken Wiltshire to the flame of his own promises

This week Ken Wiltshire, one of the two reviewers of the National Curriculum,  responded to the widespread concern about the quality and impartiality of the recently announced National Curriculum Review   Critics of school curriculum review too quick to perceive a threat instead of potential way forward.

In this article he promises a review that is professional, independent, balanced and robust based on a methodology that will be methodology comprehensive and objective.  This will include the appointment of experts  for each subject area to evaluate those components of the curriculum.

What is not to like about this?

Well there are a few problems still Mr Wiltshire.

The first most glaring problem is the other person with whom you will have to work.  Kevin Donnelly is one of the least professional, balanced or independent educational commentators I can think of in Australia today.  There is no evidence that he can or is capable of changing his spots.  But enough has already been said about this.

The second problem I have is about professionalism and respect. You tell us this will be a professional process.  Now I do assume that being professional includes being respectful.  But your use of the adjective ‘disgruntled’ to describe a letter of concern from over 150 professional educators does make we wonder.  Can you commit to respectful too perhaps?

There is one final thing Mr Wiltshire.  Would you be willing to add the word ‘open’ to your list of promises?  You encourage us all to contribute to the process but do not assure us our contributions will be made public.  Openness and transparency are features of all outstanding reviews in my book.  The Bradley Review of Higher Education and the Gonski Review are great examples of this.

A Tale of ACARA and the see-no-evil monkeys – Subtitle: There is no excuse for willful ignorance

The Australian Senate’s Education, Employment and Workplace Relations Committee is currently holding an Inquiry into the effectiveness of the National Assessment Program – Literacy and Numeracy (NAPLAN).

Over 70 submissions have been received of varying quality. In this article, I focus on the submission from the Australian Curriculum, Assessment and Reporting Authority (ACARA). ACARA is the custodian of NAPLAN and how it is used for school transparency and accountability purposes on the MySchool website.

One of the focus questions in the Inquiry’s Terms of Reference refers to the unintended consequences of NAPLAN’s introduction.  This is an important question given widespread but mainly anecdotal reports in Australia of: test cheating; schools using test results to screen out ‘undesirable enrolments’; the narrowing of the curriculum; NAPLAN test cramming taking up valuable learning time; Omega 3 supplements being marketed as able to help students perform better at NAPLAN time; and NAPLAN test booklets hitting the best seller lists.

Here is what ACARA’s submission to the Senate inquiry  has to say about this issue:

To date there has been no research published that can demonstrate endemic negative impacts in Australian schools due to NAPLAN.  While allegations have been made that NAPLAN has had unintended consequences on the quality of education in Australian schools there is little evidence that this is the case. 

The submission goes on to refer to two independent studies that investigated the unintended consequences of NAPLAN.

ACARA dismisses a Murdoch University research project[1] led by Dr Greg Thompson as flawed because its focus is on changes to pedagogical practices as a result of the existence and uses made of NAPLAN.  The basis of the dismissal is that if teaching practices change then that is all about teachers and nothing to do with NAPLAN.  Yet this report makes clear that teachers feel under pressure to make these changes, changes they don’t agree with, because of the pressures created by the use of NAPLAN as a school accountability measure.  In other words, in one clever turn of phrase, ACARA rules out of court any unintended consequences of NAPLAN that relate to changes to teachers’ practice.

ACARA also dismisses a survey undertaken by the Whitlam Institute because it  “suffers from technical and methodological limitations, especially in relation to representativeness of the samples used” and dismisses its conclusions, without  outlining detailing the findings. Now this survey was completed by nearly 8500 teachers throughout Australia and it was representative in every way (year level taught, sector, gender, years of experience) except for the oversampling in Queensland and Tasmania. In relation to this sampling concern the writers even reported that they weighted the responses to compensate for this sampling differential.  This report documents many unintended consequences that ACARA are now saying are not substantiated because of a spurious sampling critique. This is intellectually dishonest at best.

ACARA’s dismissal of all of the findings of these two research projects on spurious grounds while refraining from stating their findings is in stark contrast to its treatment of unsupported statements by non impartial stakeholders about enrolment selection concerns.

In response to the claims that some schools are using NAPLAN results to screen out ‘undesirable students’ ACARA states that it is aware of these claims but appears willing to take at face value comments from stakeholders who represent the very schools accused of unethical enrolment screening.

It is ACARA’s understanding that these results are generally requested as one of a number of reports, including student reports written by teachers, as a means to inform the enrolling school on the strengths and weaknesses of the student. The purpose for doing so is to ensure sufficient support for the student upon enrolment, rather than for use as a selection tool. This understanding is supported by media reporting of comments made by peak bodies on the subject (my emphasis).

ACARAs approach to this whole matter comes across as most unprofessional.  But, unfortunately for ACARA, this is not the whole story.  There is a history to this issue that began in 2008, almost as soon as the decision to set up MySchool was announced by the then PM Kevin Rudd as part of his school transparency agenda.

Three years ago this month I wrote an article[2] about the importance of evaluating the impact of the MySchool Website and the emergence, under FOI, of an agreement in September 2008 by State and Commonwealth Ministers of Education to:

…commence work on a comprehensive evaluation strategy for implementation, at the outset, that will allow early identification and management of any unintended and adverse consequences that result from the introduction of new national reporting arrangements for schools(my emphasis).

It was clear from the outset that this evaluation should have been managed by ACARA as the organisation established to manage the school transparency agenda.  In 2010, in response to my inquiry to ACARA on this Ministerial agreement the CEO of ACARA stated that it was not being implemented at that point in time because, early reactions to hot button issues are not useful and because the website did not yet include the full range of data planned.

This was a poor response for three reasons.

Firstly, well-designed evaluations are designed, not as afterthoughts, but as part of the development process. One of the vital elements of any useful evaluation is the collection of baseline data that would enable valid comparisons of any changes over time.  For example, information could be collected prior to the MYSCHOOL rollout on matters such as:

  • Has time allocated to non-test based subjects reduced over time?
  • Has teaching become more fact based?
  • Has the parent demographic for different schools changed as a result of NAPLAN data or student demographic data?
  • Are more resources allocated to remedial support for students who fail to reach benchmarks?
  • Are the impacts different for different types of schools?

Secondly, the commitment to evaluate was driven by concerns about the possibility of schools being shamed through media responses to NAPLAN results, the narrowing of curriculum and teaching, further residualisation of public schools, test cheating and possible negative effects on students and teachers. Identifying these concerns early would allow for revising the design elements of MySchool to mitigate the impacts in a timely fashion.  There is no real value in waiting years before deciding corrections are needed.

Thirdly, anyone who seriously believed that all of the data elements agreed as possibly in scope for MySchool was a complete list and able to be developed quickly, was dreaming.  Waiting for the full range of data meant, in reality, an indefinite delay. There are still data items in development today.

So now, five years on from the Ministerial directive that there was a need to actively investigate any unintended consequences, there is still no comprehensive evaluation in sight. One suspects that ACARA finds this quite convenient and hopes that its failure to act on this directive stays buried.

However, Ministers of Education still had concerns.  In the MCEECDYA Communiqué of April 2012 the following is reported:

Ministers discussed concerns over practices such as excessive test preparation and the potential narrowing of the curriculum as a result of the publication of NAPLAN data on My School.  Ministers requested that ACARA provide the Standing Council with an assessment of these matters.

On the basis of this statement, I wrote to ACARA on 27 April 2013 requesting information on action in response to this directive  – then over 12 months old.  To date I have received no reply.

So what sense can be made of this?

If one takes at face value the statements by ACARA that it knows of no information regarding the extent of unintended consequences one can only conclude that ACARA has twice not undertaken a Ministerial directive.

Here we have a Government body: aware of Ministers’ concerns about unintended negative consequences about a program it manages; aware of widespread anecdotal concerns, some of them quite serious; dismissing without any proper argument the few pieces of evidence that do exist; and. refusing to undertake any investigation into this matter despite two Ministerial directives to do so.

Willful ignorance about the potential unintended and harmful impacts of a program an agency has responsibility for whilst all the while professing a strong interest in this matter is highly irresponsible and unprofessional.

It is also quite astonishing given the Government’s commitment to the principle of transparency and the fact that ACARA was established specifically to bring that transparency and reporting to Australia’s schools.

But to then write a submission that almost boasts about the lack of information on this issue, while dismissing with poor arguments the evidence that is growing, is outrageous. It also gives new meaning to a throwaway line in its submission about the negative findings from the Whitlam Institute survey  “Further research with parents, students and other stakeholders is needed to confirm the survey results on well-being

Further research is indeed needed, and this further research should have been initiated by ACARA quite some time ago– five years to be precise.  It is convenient, for ACARA, that such research is not available. It is intellectually dishonest and misleading for ACARA to now state that it “takes seriously reports of unintended adverse consequences in schools. It actively monitors reports and research for indications of new issues or trends.”

Of course, there is another,  more alarming possibility, that this work has been undertaken but is not being made public, and that ACARA is misleading the Parliamentary inquiry and the public by denying that any such information exists.

In either case, I am forced to conclude that ACARA does not want any unintended consequences of a program for which it is responsible to be known, in spite of its ‘interest in this issue’ and is persisting in its position of willful ignorance.

In an effort to restore public confidence in its work, ACARA should commit to undertaking this research at arms length using independent researchers and reporting the findings to Parliament, without delay.  Perhaps this Inquiry could recommend this.