A vision for a new unified and credible approach to school assessment in Australia

 

I was only partly surprised to read in the Adelaide Advertiser[1] that Geoff Masters, CEO of the Australian Council for Educational Research (ACER) has called for the scrapping of the A-E grading system and replacing it with NAPLAN growth information.

To be blunt, I regard the A-E system as a nonsense cooked up by the previous Coalition Government and imposed on all states as a condition of funding.  It has never meant much and the different approaches to curriculum taken by the different state systems made its reporting even more confusing.

With the introduction of the Australian National Curriculum, the A-E grading system may have a more consistent approach across states but that meaning itself is often confusing and unhelpful.  As Masters notes

If a student gets a D one year and a D the next, then they might think they’re not making any progress at all when they are but the current reporting process doesn’t help them see it… [T]his could contribute to some students becoming disillusioned with the school system.

Abandoning this approach makes sense.  But the Advertiser article also implied that Masters is arguing that we should replace the A-E reporting with a NAPLAN gains process.  This to me was a complete surprise.

This is because I believe that would be a disaster and, more importantly, I am pretty sure that Masters would also see the limitations of such an approach.

At the 2010 Australian Parliamentary Inquiry into the Administration and Reporting of NAPLAN, Geoff Masters spoke at length about the limitations of NAPLAN covering the following:

  • Its limitation for students at the extremes because it is not multilevel
  • Its original purpose as a population measure and the potential reliability and validity problems with using it at school, classroom and individual student level
  • Its limited diagnostic power – because of the narrow range of testing and the multiple choice format

He also acknowledged the potential dangers of teachers teaching to the test and the narrowing of the curriculum.  (Unfortunately there appears to be a problem with the APH website and I was unable to reference this, but I have located a summary of the ACER position[2])

Now these are not minor problems.

I was also surprised because the idea that the CEO of ACER would not use this as an opportunity to talk about the benefit of diagnostic and formative assessments is unlikely. After all, these tests are important for ACER’s revenue stream.

So what is going on here?

To investigate, I decided to look beyond the Advertiser article and track down the publication that Masters was speaking to at the conference. It’s a new publication launched yesterday called Reforming Educational Assessment: Imperatives, principles and challenges[3]

And low and behold, the editor Sheradyn Holderhead got it wrong.  What Masters is arguing for is anything but the swapping out of one poorly informed reporting system (A to E Reporting) for a flawed one (NAPLAN)   He is mapping out a whole new approach to assessment that can be built on our best understandings of assessment and learning but also meet the “performativity”[4] needs of politicians and administrators.

Now some will object to the compromise taken here because they see “performativity” as a problem in and of itself.  At one level I agree but because I also look for solutions that are politically doable I tend to take a more pragmatic position.

This is because I see the reporting of NAPLAN through MySchool as a kind of one way reform – a bit like privatization of public utilities.  Once such system has been developed it is almost impossible to reverse the process.  The genie cannot be put back into the bottle.  So to me, the only solution is to build a more credible system – one that is less stressful for students, less negative for lagging students, more helpful for teachers, less likely to lead to a narrowing of the curriculum through teaching to the test and less prone to be used as a basis for school league tables.

And my take on Master’s article is that, if taken seriously, his map for developing a new assessment system would have the potential to provide the design features for a whole new approach to assessment that doesn’t require the complete overthrow of the school transparency agenda to be effective.

Here are some of the most significant points made by Masters on student assessment:

Assessment is at the core of effective teaching

Assessment plays an essential role in clarifying starting points for action. This is a feature of professional work in all fields. Professionals such as architects, engineers, psychologists and medical practitioners do not commence action without first gathering evidence about the situation confronting them. This data-gathering process often entails detailed investigation and testing. Solutions, interventions and treatments are then tailored to the presenting situation or problem, with a view to achieving a desired outcome. This feature of professional work distinguishes it from other kinds of work that require only the routine implementation of pre-prepared, one-size-fits-all solutions.

Similarly, effective teachers undertake assessments of where learners are in their learning before they start teaching. But for teachers, there are obvious practical challenges in identifying where each individual is in his or her learning, and in continually monitoring that student’s progress over time. Nevertheless, this is exactly what effective teaching requires.

Understandings derived from developments in the science of learning challenge long-held views about learning, and thus approaches to assessing and reporting learning.

These insights suggest that assessment systems need to

  • Emphasise understanding where students are at, rather than judging performance
  • Provide information about where individuals are in their learning, what experiences and activities are likely to result in further learning, and what learning progress is being made over time
  • Give priority to the assessment of conceptual understandings, mental models and the ability to apply learning to real world situations
  • Provide timely feedback in a form that a) guides student action and builds confidence that further learning is possible and b) allows learners to understand where they are in their learning and so provide guidance on next steps
  • Focus the attention of schools and school systems on the development of broader life skills and attributes – not just subject specific content knowledge
  • Take account of the important role of attitudes and self belief in successful learners

On this last point Masters goes on to say that:

Successful learners have strong beliefs in their own capacity to learn and a deep belief in the relationship between success and effort. They take a level of responsibility for their own learning (for example, identifying gaps in their knowledge and taking steps to address them) and monitor their own learning progress over time. The implications of these findings are that assessment processes must be designed to build and strengthen metacognitive skills. One of the most effective strategies for building learners’ self-confidence is to assist them to see the progress they are making.

…..  current approaches to assessment and reporting often do not do this. When students receive the same letter grade (for example, a grade of ‘B’) year after year, they are provided with little sense of the progress they are actually making. Worse, this practice can reinforce some students’ negative views of their learning capacity (for example, that they are a ‘D’ student).

Assessment is also vital in order to assess how a system is progressing – whether for a class, school, system, state or nation

Assessment, in this sense, is used to guide policy decision making or to measure the impact of interventions or treatments or to identify problems or issues

In educational debate these classroom based and the system driven assessments are often seen as in conflict and their respective proponents as members of opposing ideological and educational camps.

But the most important argument in the paper is that we have the potential to overcome the polarised approach to assessments that is typical of current discussion about education; but only if we start with the premise that the CORE purpose of assessment is to understand where students are in their learning. Other assessment goals should be built on this core.

Once information is available about where a student is in his or her learning, that information can be interpreted in a variety of ways, including in terms of the kinds of knowledge, skills and understandings that the student now demonstrates (criterion- or standards-referencing); by reference to the performances of other students of the same age or year level (norm-referencing); by reference to the same student’s performance on some previous occasion; or by reference to a performance target or expectation that may have been set (for example, the standard expected of students by the end

of Year 5). Once it is recognised that the fundamental purpose of assessment is to establish where students are in their learning (that is, what they know, understand and can do), many traditional assessment distinctions become unnecessary and unhelpful.

To this end, Masters proposes the adoption and implementation of a coherent assessment ‘system’ based on a set of 5 assessment design principles as follows

Principle 1: Assessments should be guided by, and address, an empirically based understanding of the relevant learning domain.

Principle 2: Assessment methods should be selected for their ability to provide useful information about where students are in their learning within the domain.

Principle 3: Responses to, or performances on, assessment tasks should be recorded using one or more task ‘rubrics’.

Principle 4: Available assessment evidence should be used to draw a conclusion about where learners are in their progress within the learning domain.

Principle 5: Feedback and reports of assessments should show where learners are in their learning at the time of assessment and, ideally, what progress they have made over time.

So, to return to the premise of the Advertiser article, Masters is not arguing for expanding the use value of the currently model of NAPLAN.  In fact, he is arguing for the reconceptualisation of assessment that:

  • starts with the goal of establishing where learners are in their learning within a learning domain; and
  • develops, on the basis of this a new Learning Assessment System that is equally relevant in all educational assessment contexts, including classroom diagnostic assessments, international surveys, senior secondary assessments, national literacy and numeracy assessments, and higher education admissions testing.

As the Advertiser article demonstrates, this kind of argument is not amenable to easy headlines and quick sound bytes.  Building the support for moving in this direction will not be easy.

But the first step is to recognize that the popular understanding that system based assessment and ‘classroom useful’ assessment are and must necessarily be at cross purposes and to start to articulate how a common approach could be possible.  Masters refers to this as the unifying principle:

….. it has become popular to refer to the ‘multiple purposes’ of assessment and to assume that these multiple purposes require quite different approaches and methods of assessment. …

This review paper has argued …. that assessments should be seen as having a single general purpose: to establish where learners are in their long-term progress within a domain of learning at the time of assessment. The purpose is not so much to judge as to understand. This unifying principle, which has potential benefits for learners, teachers and other educational decision-makers, can be applied to assessments at all levels of decision-making, from classrooms to cabinet rooms.

So if you are still not convinced that Masters is NOT arguing for replacing the A-E reporting with NAPLAN growth scores, this quote may help:

As long as assessment and reporting processes retain their focus on the mastery of traditional school subjects, this focus will continue to drive classroom teaching and learning. There is also growing recognition that traditional assessment methods, developed to judge student success on defined bodies of curriculum content, are inadequate for assessing and monitoring attributes and dispositions that develop incrementally over extended periods of time.


[4] This is a widely used term usually associated with the work of Stephen J. Ball. In simple terms it refers to our testing mania in schools and the culture and conceptual frameworks that support reform built around testing data.  To read more this might be a useful starting point http://www.scribd.com/doc/70287884/Ball-performativity-teachers

Advertisements

The Scourge of School League Tables

 

I was stunned to see that the Canberra Times published its own league table about ACT Schools.  I mean I was sure that a respectable rag like the CT would know better than to engage in a cheap stunt like this

 

Sadly I was wrong.

 

Thankfully Trevor Cobbold of Save our Schools fame has stepped in and provided a telling commentary here Save Our Schools Canberra: The Whackiness of School League Tables

 

The table shows the following

 

  1. That the data from NAPLAN at the schools level is completely meaningless and unreliable at least when it comes to drawing any conclusions about school or teacher quality.
  2. Across year groups, across the disciplines, and across the independent, Catholic and government sectors, schools are jumping around all over the place!

 

The simple fact is that student cohorts change every year. And the smaller the school, the greater the chance of wild fluctuations.

 

You see NAPLAN was never ever designed to be reliable and valid at the individual school level  – never.  It is/was designed a population measure and at that level and that level only it is quite reliable and useful.  At the school level – not so much

 

The decision to provide NAPLAN results at the school level is a political decision and there is no evidence that the results are valid at this level – they were not intended to be used in this way.

 

According to Cobbold in the ACT In Year 3 writing, one school went from 1st last year to 66th this year, whilst in Year 3 grammar another school went from 81st to 4th!

 

These are not aberrations, as similarly spectacular rises and falls appear throughout the tables.

 

So what if anything d these league table results tell us:

 

“what really matters – attracting and retaining the best in teaching, giving schools and systems the support they need to become hubs of collaborative professional learning, and improving equity by targeting resources to students who need extra assistance, as recommended by the Gonski report into school funding”.

 

 

 

It really is as simple at that

 

Teaching about Human Rights: What is its DNA?

During the consultation phase on the development of the Australian National Curriculum, the Australian Human Rights Commission stated that they were concerned about the lack of a comprehensive and coherent coverage of Human Rights in the Draft Curriculum.  They also indicated on their website that

The Commission is participating in consultations on the draft curriculum and recommending ways in which the human rights content in the curriculum can be strengthened.

They also posted position paper on  how it could best be included.    Position Paper

The fact that their coverage of this matter on their website today still states that they are ‘engaged in consultations’ suggests that their intervention may have been too late.

However, for those teaching older students, who would like to include the study of the ideas and issues surrounding human rights in their teaching this video developed by the London School of Economics  The Burning Issue: The DNA of Human Rights | British Politics and Policy at LSE.  might be a useful resource.  It will certainly promote an interesting discussion and draw on important  ideas.

I was particularly drawn to the way in which the speaker, Professor Conor Gearty, demonstrated through interviews and props how

All powerful emancipatory ideas get sucked into the vortex of power, which seeks, not to remove them, but to twist them to meanings that suit the powerful

And another US Ed Reform Fight-back Group is also announced

Meanwhile also in the US another ed reform fight-back campaign is launched

On 7 march 2013 Diane Ravitch launched a new network devoted to the defense and improvement of public education in the US called the Network for Public Education.

Its goal is to bring together grassroots activists and organizations from around the country, and endorse candidates for office, with the common goal of protecting and strengthening our public schools.

Diane Ravitch said

The Network for Public Education will give voice to the millions of parents, educators, and other citizens who are fed up with corporate-style reform. We believe in community-based reform, strengthening our schools instead of closing them, respecting our teachers and principals instead of berating them, educating our children instead of constantly testing them. Our public schools are an essential democratic institution. We look forward to working with friends and allies in every state and school district who want to preserve and improve public education for future generations.”

Our nation’s schools are at a crossroads. Wealthy individuals are pouring unprecedented amounts of money into state and local school board races, often into places where they do not reside, to elect candidates intent on undermining and privatizing our public schools. The Network for Public Education will collaborate with other groups and organizations to strengthen our public schools in states and districts throughout the nation, share information and research about what works and what doesn’t work, and endorse and grade candidates based on our shared commitment to the well-being of our children, our society, and our public schools. We will help candidates who work for evidence-based reforms and who oppose high-stakes testing, mass school closures, the privatization of our public schools and the outsourcing of core academic functions to for-profit corporations.

Lets hope these two new networks can collaborate and build on each others’ strengths.

To find out more visit http://dianeravitch.net/2013/03/07/breaking-news-new-group-to-oppose-corporate-reforms/

Education Opportunity Network

For readers interested in education reform fight-back initiatives the latest US appear to be the Education Opportunity Network.  

This initiative aims to make sure all children and youth have the opportunity to learn by sharing provocative information, analysis, and opportunities for action — and by linking educators in the trenches, students, parents, community leaders, education experts, and progressive activists who know education is crucial to the fight for democracy and economic progress. 

Education Opportunity Network.

Girls Not Brides: Traditions can change – ending child marriage – YouTube

Girls Not Brides: Traditions can change – ending child marriage – YouTube.

This video neatly encapsulates why just ending child marriage could do so much in a development context – increase literacy,  reduce mother and child mortality, reduce poverty and lack of earning power.

One area of child marriage consistently overlooked is within  Australia’s remote Indigenous communities.  They have conditions quite similar to women in developing world’s failed state situations.  However because they live in one of the wealthiest developed countries in the world, the lens of development is never applied.

This is not just about traditions of arranged marriages – which still take place but the accepted practice of girls leaving school to have babies at a very  tender age.  Investing in educating girls about sexual health and sexuality  and in persuading girls and their families of different possible futures could make such a difference to communities.

We have well funded football programs right across North Australia because we see this as a way to persuade remote Indigenous boys of the benefits of fitness, health, education and having goals.  When I asked why there was not any equivalent program for girls I invariably received one of the following 2 answers

“if we can get boys to attend school the  girls will follow” or ” but girls can be part of the program too.

NOT GOOD ENOUGH

Do you want to be on the right side of history? Join the Gonski Week of Action 10–17 March 2013

Its not to late you know.  There is still time to join with all those Australians who are saying to Governments – implement Gonski now.

There are just a few weeks to go until politicians will meet to decide on schools funding.  A few critical weeks – they need to know that support for this is loud and strong. This is a critical time for the Gonski campaign.

Because of this critical timeframe the Campaign is being stepped up, with a National Gonski Week of Action.

There are 5 important ways to show your support for Gonski: Go to http://igiveagonski.com.au/2013-gonski-week-action/ and be part of history