Home » Education Accountability
Category Archives: Education Accountability
Teacher Self Reflection Tools: a double edged sword
I have been pleasantly surprised with the work the Australian Institute of Teaching and School Leadership (AITSL) has undertaken to develop a number of seemingly high quality, well tested and useful self reflection and learning tools for teachers to support AITSL’s core work of building the capacity of teachers and school leaders. For example The 360 student feedback tools[1] the teacher standards illustrations of practice[2] and the teacher self assessment[3] tools all have real potential to be useful for teachers who are taking responsibility for their own learning and development in schools that support and encourage collaboration, mentoring and peer support.
In fact I would like to suggest that the work of AITSL has the potential to be a very important counter point to all the US borrowed corporate reforms represented by NAPLAN, Performance pay and all the rest.
But to be effective the work of AITS needs to be able to stand apart from all the less worthy reforms. The self-reflective tools are a very good example of these challenges. If they can be kept apart from the evaluation, performance management tendencies of corporate reform and be quarantined for the use by teachers and their schools for authentic professional learning, they have the potential to be very significant tools for building collective teacher capacity.
If however they are captured to be used as part of the new performance management practices that are being imposed on teachers, all the wonderful work involved in developing them will go down the toilet.
Anthony Cody talks about these same tensions in the US context. In a recent blog[4] he responds to a Bill Gate TEDX talk on the value of videos in classrooms. According to Cody, Bill Gates rationale for promoting video cameras in schools goes as follow
… there’s one group of people that get almost no systematic feedback to help them do their jobs better. Until recently, 98% of teachers just got one word of feedback: “satisfactory.” Today, districts are revamping the way they evaluate teachers. But we still give them almost no feedback that actually helps them improve their practice. Our teachers deserve better. The system we have today isn’t fair to them. It’s not fair to students, and it’s putting America’s global leadership at risk.
Cody notes that Gates slides from feedback to evaluation without pause as though they are one and the same.
Do you notice something? He starts out talking about feedback, but then slides into describing a formal evaluation process. There are LOTS of ways to enhance feedback that could have nothing at all to do with our evaluation systems ….
They are not. There is a world of difference between:
- Professional learning: as teachers working together, observing each others practice; using tools that give them information about their practice for them to use as they see fit; reflecting on their practice alone or in teams; trialling changes; reflecting; and giving mutual feedback; and
- Performance review: where external parties apply standards to an assessment of practice
The problem is that as soon as a tool is captured for use for the second purpose – performance review – the less likely it is that teachers will trust it and see it as useful.
But this slide happens all the time. And we are in danger of this happening with the tools developed by AITSL. This is because we are focusing on the wrong things. The Commonwealth Government tells us that what we need is a national best practice performance management framework and high quality tools.
Linda Darling Hammond on the other had argues that it is not a good framework that is lacking. Rather what we lack, is time – time in schools for teachers to collaborate, to work with others to reflect on their practice and a culture where this is expected not as a fearful evaluation process but as an integral part of professional development
As I see it the work of AITSL could go either way and I just hope that it is possible to corral some of the best of their work and make sure it is not captured to serve the performativity agenda, For as Anthony Cody says:
Right beneath the surface are these seeds of possibility, waiting for the right conditions to come about. You take an area, a school, a district, you change the conditions, give people a different sense of possibility, a different set of expectations, a broader range of opportunities, you cherish and value the relationships between teachers and learners, you offer people discretion to be creative and to innovate in what they do, and schools that were once bereft spring to life.
Kevin Donnelly thinks that Fabianism is a dirty word.
We’ve put up with absolute rubbish from Kevin Donnelly for too long. It’s time to look at his claims without the emotion and invective
In his latest rant, in The Australian, called, “Education saviour is pulling too many levers[1]”, Donnelly makes the following claims.
1. Julia Gillard “in a desperate attempt” is going to use education as her lever to stay in power
Sadly, and a little reluctantly, I share concerns about the growing centrality of education in the future election debate. Although chances are slim, I am pinning my hopes on progress on implementing the key components of the Gonski reforms prior to the election to the extent that they cannot easily be rolled back.
The temptation to use it the Gonski implementation plan as an election carrot will not save the ALP but it will cost public schools dearly.
2. Billions have been wasted on the Building the Education Revolution program that forced off-the-shelf, centrally mandated infrastructure on schools with little, if any, educational benefit;
Donnelly clearly has not read the ANAO Audit report into the BER[2], because it concludes that where there were poor decisions and centralized rollouts the culprits were state Governments not the Commonwealth and that to some extent this was inevitable given the justifiable time constraints. May I also remind him that this was a GFC response first and foremost not an education initiative? The audit report makes this clear:
The Government decided on school based infrastructure spending because it had a number of elements that supported stimulus objectives
It also notes that:
The objectives of the BER program are, first, to provide economic stimulus through the rapid construction and refurbishment of school infrastructure and, second, to build learning environments to help children, families and communities participate in activities that will support achievement, develop learning potential and bring communities together[3]
For many schools the capital works were a godsend because the new hall or learning space gave them the capacity to do the thing that Donnelly most encourages – use new space to increase local innovative solutions to education challenges. Indeed the audit report noted that over 95% of principals that responded to the ANAO survey indicated that the program provided something of ongoing value to their school and school community.[4]
3. The computers in schools program delivered thousands and thousands of now out-of-date computers that schools can ill-afford to maintain or update.
I am not one to argue that ICT is the magic bullet answer to everything about teaching and learning in our schools. However I am convinced that with well-informed computer literate teachers, who are also good teachers in the broader sense, students can only benefit. I also acknowledge that a high level of computer literacy is now a core area of learning. To achieve this even “out of currency” computer hardware will be better than no computers
Any ICT hardware rollout will result in out-of-date computers and a maintenance/update impost. But the state of ICT infrastructure in our schools desperately needed to be addressed. Is Donnelly really arguing that schools that do not have enough in their budgets to manage the whole-of-life costs of having computers should go without? I wonder which schools these might be?
4. Julia Gillard’s data fetish is forcing a centralised and inflexible accountability regime on schools, government and non-government, that is imposing a command and control regime on classrooms across the nation.
There is no doubt that we could benefit from a better accountability and reporting regime – for all schools. So this is one of the few areas where Donnelly and I have aligned concerns but possibly for different reasons. I continue to believe that the changes to the original intention of NAPLAN testing has been disastrous for some Australian schools – but possibly not the ones dear to Donnelly’s heart.
The reporting of NAPLAN results at the school level has, almost certainly, distorted what is taught in schools[5]. This is especially the case in schools where students struggle – our highly concentrated low SES schools. It has also contributed to the residualisation of the public school system. And we now have evidence that when the middle class students are leached out of public schools, public school students loose out in lots of ways. For example they lose out because of the loss of articulate and ‘entitled’ parent advocates for the needs of the schools. But they also lose out because each middle class child is actually a resource. That is their existence in the class enhances the learning of all students in that class.[6].
Donnelly, on the other hand, appears to be more concerned that non-Government schools are now under the same reporting obligations as government schools. I know of no other area of Commonwealth funding that was not expected to provide a defined level of accountability and reporting. This anomaly was way overdue.
5. The Gillard-inspired national curriculum, instead of embracing rigorous, academic standards, is awash with progressive fads such as child-centred, inquiry-based learning, all taught through a politically correct prism involving indigenous, Asian and environmental perspectives.
Donnelly appears to have a short memory on this matter. The national curriculum effort was kicked off by the previous Howard Government – and that is why History was singled out above other social science disciplines.
Perhaps Donnelly has not read the national curriculum? If he had he would know that it is just a sequence and scoping exercise and does not address pedagogy at all. Donnelly has had a bee in his bonnet for years about so called ‘progressive fads’ based on nothing more than sheer ignorance. And as for the cross curriculum perspectives – these came out of extensive consultation and negotiation and were not imposed by the Gillard Government. While there are unfortunately many examples of Commonwealth overreach, the cross-curricular perspectives are not examples.
6. Even though the Commonwealth Government neither manages any schools nor employs any teachers, Gillard is making it a condition of funding that every school across Australia must implement Canberra’s (sic) National Plan for School Improvement.
This is another area where, to some extent, I do agree with Donnelly but for very different reasons.
My position is that the National Plan for School Improvement is Commonwealth overreach that was unnecessary and risky because it could have put the Gonski implementation at risk.
The National Plan for School Improvement was unnecessary because, all education systems throughout the country already had some form of school improvement planning and annual reporting, and had begun to share good practice through the National Partnership process. It was also unnecessary because it foolishly cut across the more informed and consultative process being undertaken by AITSL to grow the teacher performance feedback and improvement process in collaboration with the various teaching institutes around Australia. This process had a strong emphasis on supporting teacher development and self-reflection based on well-supported peer, supervisor and student feedback. The Commonwealth initiative has recast the whole process into a high stakes, external reporting context that will be much less useful and teacher friendly. This is a pity. AITSL’s work should not have been distorted in this manner.
It was, and is, risky as some states seized on the obligations of the Plan as the rationale to push back on the Gonski reforms. Tying the two together was poor strategy, in light of the importance of implementing Gonski between now and September 2013.
Donnelly’s objection to the Plan appears to be that is is imposed on the non Government sectors that should, according to Donnelly, be able to receive significant levels of Commonwealth funding with no accountability?. It’s the imposts he objects to, not their design elements.
7. Research here and overseas proves that the most effective way to strengthen schools, raise standards and assist teachers is to embrace diversity, autonomy and choice in education. The solution lies in less government interference and micro-management, not more.
I am afraid that Donnelly’s claims that autonomy and choice is the best way to strengthen schools does not have a shred of evidence. I, and others, have written about the autonomy claims[7] and there is now solid international evidence confirming that market models of education choice are disastrous for education equity and therefore for education overall[8].
8. Autonomy in education helps to explain why Catholic and independent schools, on the whole, outperform government schools.
There is now enduring evidence that the differences in school outcomes are overwhelmingly connected with student demography and not schooling system. When SES is taken into account the non Government systems do not perform any better at all. The very detailed research undertaken by Richard Teese[9] in the context of the Gonski Review process concluded that:
Using NAPLAN data, the paper shows that public schools work as well or better than private schools (including Catholic schools). This finding echoes the results of PISA 2009 that, after adjustment for intakes, public schools are as successful as private schools
9. Gillard’s plan for increased government regulation and control and a one size fits all, lowest common denominator approach is fabianism and based on the socialist ideal of equality of outcomes.
Now this is the strangest claim of all. Here Donnelly uses fabianism as a slur and it is not the first time he has taken this tack. However it is a term so quaint, so rarely used, that this tactic may well pass unnoticed. In fact in order to find a useful definition I had to go back to 1932 to an essay by GDH Cole[10]. Cole’s explanation is interesting given the implied nastiness of fabianism:
Whereas Marxism looked to the creation of socialism by revolution based on the increasing misery of the working class and the breakdown of capitalism through its inability to solve the problem of distribution, Webb argued that the economic position of the workers had improved in the nineteenth century, was still improving and might be expected to continue to improve. He regarded the social reforms of the nineteenth century (e.g. factory acts, mines acts, housing acts, education acts) as the beginnings of socialism within the framework of capitalist society. He saw legislation about wages, hours and conditions of labor, and progressive taxation of capitalist incomes as means for the more equitable distribution of wealth; …
And
The Fabians are essentially rationalists, seeking to convince men by logical argument that socialism is desirable and offering their arguments to all men without regard to the classes to which they belong. They seem to believe that if only they can demonstrate that socialism will make for greater efficiency and a greater sum of human happiness the demonstration is bound to prevail.
So our progressive tax system, our Fair Work Australia, our transfer payments to those in poverty, our national health system, our public education system, our welfare safety net, our superannuation minimums – these are all examples of fabianism at work, not because fabianism is a secret sect with mal intent as implied by Donnelly but because we have come to see the benefits of a strong cohesive society where the wealth of the country is not enjoyed by the few while the majority slave in misery.
What’s so bad about our proud achievements Donnelly? I for one want to keep moving in this direction and for me implementing the Gonski reform is the essential next step in schooling policy.
10. Tony Abbott’s view of education, is based on diversity and choice where schools are empowered to manage their own affairs free from over regulation and constraint.
It is interesting that Donnelly thinks he knows what Tony Abbott’s view of education is, because I suspect most of us remain unclear on this matter. Abbott has said on one occasion that more funding should go to Independent schools – an astonishing claim given our profile relative to all other countries. His shadow Minister has said a bit more but his statement that we should go back to didactic teaching (like when he was a boy) does not imply a commitment to allowing schools to manage their own affairs to me. But maybe he only means that this is what Government schools should do. That would probably be OK according Kevin Donnelly’s view of the world.
[1] http://www.theaustralian.com.au/national-affairs/opinion/education-saviour-is-pulling-too-many-levers/story-e6frgd0x-1226610421025
[3] Ibid P 8
[4] Ibid P 26
[5] A useful, research article about this is the submission prepared by Dr Greg Thompson in response to the Parliamentary Inquiry into the Australian Education Bill 2012 – Submission no. 16 available at this URL http://www.aph.gov.au/Parliamentary_Business/Committees/House_of_Representatives_Committees?url=ee/auseducation/subs.htm
[6]The best explanation for the important of ‘ other student affect’ on student learning is from an unpublished paper by Chris Bonner where he notes that “the way this resource of students is distributed between schools really matters. Regardless of their own Socio-economic background, students attending schools in which the average socio economic background is high tent to perform better that if they are enrolled in a school with below Socio-economic intake
[7] See for example, http://educatorvoices.wordpress.com/2012/03/27/school-autonomy-its-a-system-thing/ and http://www.saveourschools.com.au/media-releases/media-release-school-autonomy-is-not-the-success-claimed
[8]The problems with school choice
Choice and autonomy is questioned by OECD’s Andreas Schleicher
http://www.huffingtonpost.com/andreas-schleicher/learning-from-britain-abo_b_2646150.html
A vision for a new unified and credible approach to school assessment in Australia
I was only partly surprised to read in the Adelaide Advertiser[1] that Geoff Masters, CEO of the Australian Council for Educational Research (ACER) has called for the scrapping of the A-E grading system and replacing it with NAPLAN growth information.
To be blunt, I regard the A-E system as a nonsense cooked up by the previous Coalition Government and imposed on all states as a condition of funding. It has never meant much and the different approaches to curriculum taken by the different state systems made its reporting even more confusing.
With the introduction of the Australian National Curriculum, the A-E grading system may have a more consistent approach across states but that meaning itself is often confusing and unhelpful. As Masters notes
If a student gets a D one year and a D the next, then they might think they’re not making any progress at all when they are but the current reporting process doesn’t help them see it… [T]his could contribute to some students becoming disillusioned with the school system.
Abandoning this approach makes sense. But the Advertiser article also implied that Masters is arguing that we should replace the A-E reporting with a NAPLAN gains process. This to me was a complete surprise.
This is because I believe that would be a disaster and, more importantly, I am pretty sure that Masters would also see the limitations of such an approach.
At the 2010 Australian Parliamentary Inquiry into the Administration and Reporting of NAPLAN, Geoff Masters spoke at length about the limitations of NAPLAN covering the following:
- Its limitation for students at the extremes because it is not multilevel
- Its original purpose as a population measure and the potential reliability and validity problems with using it at school, classroom and individual student level
- Its limited diagnostic power – because of the narrow range of testing and the multiple choice format
He also acknowledged the potential dangers of teachers teaching to the test and the narrowing of the curriculum. (Unfortunately there appears to be a problem with the APH website and I was unable to reference this, but I have located a summary of the ACER position[2])
Now these are not minor problems.
I was also surprised because the idea that the CEO of ACER would not use this as an opportunity to talk about the benefit of diagnostic and formative assessments is unlikely. After all, these tests are important for ACER’s revenue stream.
So what is going on here?
To investigate, I decided to look beyond the Advertiser article and track down the publication that Masters was speaking to at the conference. It’s a new publication launched yesterday called Reforming Educational Assessment: Imperatives, principles and challenges[3]
And low and behold, the editor Sheradyn Holderhead got it wrong. What Masters is arguing for is anything but the swapping out of one poorly informed reporting system (A to E Reporting) for a flawed one (NAPLAN) He is mapping out a whole new approach to assessment that can be built on our best understandings of assessment and learning but also meet the “performativity”[4] needs of politicians and administrators.
Now some will object to the compromise taken here because they see “performativity” as a problem in and of itself. At one level I agree but because I also look for solutions that are politically doable I tend to take a more pragmatic position.
This is because I see the reporting of NAPLAN through MySchool as a kind of one way reform – a bit like privatization of public utilities. Once such system has been developed it is almost impossible to reverse the process. The genie cannot be put back into the bottle. So to me, the only solution is to build a more credible system – one that is less stressful for students, less negative for lagging students, more helpful for teachers, less likely to lead to a narrowing of the curriculum through teaching to the test and less prone to be used as a basis for school league tables.
And my take on Master’s article is that, if taken seriously, his map for developing a new assessment system would have the potential to provide the design features for a whole new approach to assessment that doesn’t require the complete overthrow of the school transparency agenda to be effective.
Here are some of the most significant points made by Masters on student assessment:
Assessment is at the core of effective teaching
Assessment plays an essential role in clarifying starting points for action. This is a feature of professional work in all fields. Professionals such as architects, engineers, psychologists and medical practitioners do not commence action without first gathering evidence about the situation confronting them. This data-gathering process often entails detailed investigation and testing. Solutions, interventions and treatments are then tailored to the presenting situation or problem, with a view to achieving a desired outcome. This feature of professional work distinguishes it from other kinds of work that require only the routine implementation of pre-prepared, one-size-fits-all solutions.
Similarly, effective teachers undertake assessments of where learners are in their learning before they start teaching. But for teachers, there are obvious practical challenges in identifying where each individual is in his or her learning, and in continually monitoring that student’s progress over time. Nevertheless, this is exactly what effective teaching requires.
Understandings derived from developments in the science of learning challenge long-held views about learning, and thus approaches to assessing and reporting learning.
These insights suggest that assessment systems need to
- Emphasise understanding where students are at, rather than judging performance
- Provide information about where individuals are in their learning, what experiences and activities are likely to result in further learning, and what learning progress is being made over time
- Give priority to the assessment of conceptual understandings, mental models and the ability to apply learning to real world situations
- Provide timely feedback in a form that a) guides student action and builds confidence that further learning is possible and b) allows learners to understand where they are in their learning and so provide guidance on next steps
- Focus the attention of schools and school systems on the development of broader life skills and attributes – not just subject specific content knowledge
- Take account of the important role of attitudes and self belief in successful learners
On this last point Masters goes on to say that:
Successful learners have strong beliefs in their own capacity to learn and a deep belief in the relationship between success and effort. They take a level of responsibility for their own learning (for example, identifying gaps in their knowledge and taking steps to address them) and monitor their own learning progress over time. The implications of these findings are that assessment processes must be designed to build and strengthen metacognitive skills. One of the most effective strategies for building learners’ self-confidence is to assist them to see the progress they are making.
….. current approaches to assessment and reporting often do not do this. When students receive the same letter grade (for example, a grade of ‘B’) year after year, they are provided with little sense of the progress they are actually making. Worse, this practice can reinforce some students’ negative views of their learning capacity (for example, that they are a ‘D’ student).
Assessment is also vital in order to assess how a system is progressing – whether for a class, school, system, state or nation
Assessment, in this sense, is used to guide policy decision making or to measure the impact of interventions or treatments or to identify problems or issues
In educational debate these classroom based and the system driven assessments are often seen as in conflict and their respective proponents as members of opposing ideological and educational camps.
But the most important argument in the paper is that we have the potential to overcome the polarised approach to assessments that is typical of current discussion about education; but only if we start with the premise that the CORE purpose of assessment is to understand where students are in their learning. Other assessment goals should be built on this core.
Once information is available about where a student is in his or her learning, that information can be interpreted in a variety of ways, including in terms of the kinds of knowledge, skills and understandings that the student now demonstrates (criterion- or standards-referencing); by reference to the performances of other students of the same age or year level (norm-referencing); by reference to the same student’s performance on some previous occasion; or by reference to a performance target or expectation that may have been set (for example, the standard expected of students by the end
of Year 5). Once it is recognised that the fundamental purpose of assessment is to establish where students are in their learning (that is, what they know, understand and can do), many traditional assessment distinctions become unnecessary and unhelpful.
To this end, Masters proposes the adoption and implementation of a coherent assessment ‘system’ based on a set of 5 assessment design principles as follows
Principle 1: Assessments should be guided by, and address, an empirically based understanding of the relevant learning domain.
Principle 2: Assessment methods should be selected for their ability to provide useful information about where students are in their learning within the domain.
Principle 3: Responses to, or performances on, assessment tasks should be recorded using one or more task ‘rubrics’.
Principle 4: Available assessment evidence should be used to draw a conclusion about where learners are in their progress within the learning domain.
Principle 5: Feedback and reports of assessments should show where learners are in their learning at the time of assessment and, ideally, what progress they have made over time.
So, to return to the premise of the Advertiser article, Masters is not arguing for expanding the use value of the currently model of NAPLAN. In fact, he is arguing for the reconceptualisation of assessment that:
- starts with the goal of establishing where learners are in their learning within a learning domain; and
- develops, on the basis of this a new Learning Assessment System that is equally relevant in all educational assessment contexts, including classroom diagnostic assessments, international surveys, senior secondary assessments, national literacy and numeracy assessments, and higher education admissions testing.
As the Advertiser article demonstrates, this kind of argument is not amenable to easy headlines and quick sound bytes. Building the support for moving in this direction will not be easy.
But the first step is to recognize that the popular understanding that system based assessment and ‘classroom useful’ assessment are and must necessarily be at cross purposes and to start to articulate how a common approach could be possible. Masters refers to this as the unifying principle:
….. it has become popular to refer to the ‘multiple purposes’ of assessment and to assume that these multiple purposes require quite different approaches and methods of assessment. …
This review paper has argued …. that assessments should be seen as having a single general purpose: to establish where learners are in their long-term progress within a domain of learning at the time of assessment. The purpose is not so much to judge as to understand. This unifying principle, which has potential benefits for learners, teachers and other educational decision-makers, can be applied to assessments at all levels of decision-making, from classrooms to cabinet rooms.
So if you are still not convinced that Masters is NOT arguing for replacing the A-E reporting with NAPLAN growth scores, this quote may help:
As long as assessment and reporting processes retain their focus on the mastery of traditional school subjects, this focus will continue to drive classroom teaching and learning. There is also growing recognition that traditional assessment methods, developed to judge student success on defined bodies of curriculum content, are inadequate for assessing and monitoring attributes and dispositions that develop incrementally over extended periods of time.
[1] http://www.adelaidenow.com.au/news/south-australia/educational-researcher-calls-for-reform-of-school-assessment-system/story-e6frea83-1226600163056
[4] This is a widely used term usually associated with the work of Stephen J. Ball. In simple terms it refers to our testing mania in schools and the culture and conceptual frameworks that support reform built around testing data. To read more this might be a useful starting point http://www.scribd.com/doc/70287884/Ball-performativity-teachers
The Scourge of School League Tables
I was stunned to see that the Canberra Times published its own league table about ACT Schools. I mean I was sure that a respectable rag like the CT would know better than to engage in a cheap stunt like this
Sadly I was wrong.
Thankfully Trevor Cobbold of Save our Schools fame has stepped in and provided a telling commentary here Save Our Schools Canberra: The Whackiness of School League Tables
The table shows the following
- That the data from NAPLAN at the schools level is completely meaningless and unreliable at least when it comes to drawing any conclusions about school or teacher quality.
- Across year groups, across the disciplines, and across the independent, Catholic and government sectors, schools are jumping around all over the place!
The simple fact is that student cohorts change every year. And the smaller the school, the greater the chance of wild fluctuations.
You see NAPLAN was never ever designed to be reliable and valid at the individual school level – never. It is/was designed a population measure and at that level and that level only it is quite reliable and useful. At the school level – not so much
The decision to provide NAPLAN results at the school level is a political decision and there is no evidence that the results are valid at this level – they were not intended to be used in this way.
According to Cobbold in the ACT In Year 3 writing, one school went from 1st last year to 66th this year, whilst in Year 3 grammar another school went from 81st to 4th!
These are not aberrations, as similarly spectacular rises and falls appear throughout the tables.
So what if anything d these league table results tell us:
“what really matters – attracting and retaining the best in teaching, giving schools and systems the support they need to become hubs of collaborative professional learning, and improving equity by targeting resources to students who need extra assistance, as recommended by the Gonski report into school funding”.
It really is as simple at that
Don’t be fooled: Pyne’s NAPLAN proposal is worse – much worse
The response of the press to the Media Release My School the source of NAPLAN angst – Liberal Party of Australia by Christopher Pyne, Shadow Minister for Education, announcing that the Liberal Party have heard all the concerns of educators and parents about the danger of publishing NAPLAN results so they will stop it, shows us everything that is wrong with the press in Australia.
The press passed the contents of this press release on with absolutely no analysis whatsoever. A supposedly good news story. The Liberal Party is listening and responding. Wrong. Their tears of concern are but crocodile tears.
What Pyne actually announced is that the Liberal Party will stop publishing the NAPLAN raw scores and would start publishing school improvement measures. This will lead to the same pressure, but draw on different rubbish data.
Those of you who keep up with the education reform policy debates in the US might know improvement measures by other names:
– AYP – Adequate Yearly Progress or
- VAM – Value Added Measures
Do these terms start to ring any bells? They should. AYP measures were used by Joel Klein in NYC schools to target schools for closure and set up in their place – often on the same site - Charter schools. This has been incredibly disruptive for the families suddenly left with no local public school (often Charters selected students by application from parents and lotteries) and there is no peer-reviewed research that shows sustainable learning improvements.
VAM measures are now being used across the US to assess teacher quality even though no reputable psychometrician will confirm that the national testing results – at the classroom level - have any validity or reliability. Teachers’ futures are made or broken on the basis of this sort of dodgy data dealing.
It doesn’t take a conspiracy minded person to join the dots. Of course VAM and or AYP would be good for the LNP. They can use it to adopt the disastrous Ed Reform policies of the US – to destroy teacher conditions and the power of unions, to close public schools and set up private schools using public school funds and so on.
I have another concern about Value Added Measures. With Value Added Measures the actual results of a school are not relevant. All that is measured is the growth in student learning from one year to the next. Now some argue that this is good for struggling schools because it will end the ‘shame job’. Arguably a high performing school that is resting on its laurels could end up exposed and a low performing school that still has well below average results but is improving can look good. This is good, surely?
How is a piece of data that says, “This [low performing] school [in a low SES area] is doing a very good job. The student learning outcomes are excellent [for these students], ever justifiable?
One of the few good things that has come out of this whole NAPLAN debacle is that it gave teeth and exposure to the work by equity researchers and activists like Professor Richard Teese, Chris Bonner and Bernie Sheperd. Their work saw the light of day through the Gonski review process and importantly could no longer be disputed. Their research changed the Gonski debate – there is no doubt about this.
Now don’t for a moment think that I am justifying the publication of NAPLAN results at school level. In fact strangely enough the equity research referred to above was impeded as much as it was aided by the Myschool data because of the format of and level at which it is presened. Myschool won’t allow the data to be manipulated or rolled up. But their analysis relied on rolling up the data so that groups of schools (e.g. low SES schools, rural schools, etc) could be compared to other groups of schools. This is not possible using Myschool and researchers had to go to great lengths to get around this problem.
If I was a defender of continuing high levels of Government funding to the schools who need it least it would be in my interest to make the raw scores, that fed the work of equity campaigners, simply disappear. Without it we could return to the she said he said debates about equity in Australia.
The raw NAPLAN scores have proven once and for all that demography still is destiny in today’s Australia. They also prove that two children of equal social background going to different schools will have different student learning outcomes because of our highly socially segregated schooling system. This is known as ‘ the school effect’ and Australia leads the way in this area, to our shame.
I will continue to oppose the way in which we use NAPLAN scores at school level, but I will continue to fight for the data about children’s learning by student demography, by school type and so on, to be available for equity research. NAPLAN is not the best data, but that is a whole other debate that we wont get to have if we reduce NAPLAN raw scores to Value Added Measures.
I oppose the idea of schools being wholly accountable for the progress of their students without any support that recognises their unequal challenges, but I will continue to fight for the notion that Governments should be accountable, to the public, as citizens – not just as parents for providing a high quality education with equal opportunity for all. Pyne’s proposal will kill the data available to support this, but wont stop the negative NAPLAN effects.
Gonski – The poisonous pen of the IPA strikes again
Julie Novak’s article, Gonski report too narrowly focused shows once again just how little the IPA and its ilk give a dam for ‘other peoples children’.
But her statement that the Australian Education Union (AEU) is pushing for more TAXPAYER funding for GOVERNMENT schools for entirely self-interested reasons is beyond unacceptable – it is disgraceful.
The Australian Education Union has long called for the Gonski recommendations to be implemented, and it is not difficult to understand why. Additional taxpayer funding for government schools would further entrench teachers’ employment, and provide opportunities for the union to skim some of the extra funds via higher teacher salaries in any future negotiations with the states.
Teaching conditions have an impact on classroom learning conditions. To think otherwise is idiocy. Julie could ask the Independent Education Union of Australia for ther views on this link. The AEU have an interest in increasing funding for Government schools because they work in them and know the struggles and challenges involved in delivering high quality education in a cash starved environment that unequally serves the needs of the vast majority of our most needy students.
But we should also keep in mind that the AEU is the most important public school public school advocacy group that we have in this country and they wield nothing like the power of the independent schools lobby – the Independent Schools Council of Australia (ICSA) with the very outspoken and powerful ex senior public servant Bill Daniels as its Executive Director
Unlike independent schools, public schools do not have a Bill Daniel equivalent. The ‘caretaker owners’ of government schools are governments, and oddly enough they cant lobby themselves. When an education system was established in the ACT there was such a body – the ACT Schools Authority – set up to be independent from Government and able to advocate for schools. But it did not last the first major budget cut.
And of course the Independent Schools Association are not at all self interested. They are seeking an increase in funding for the Independent sector because this is the most important priority for Australia today and this is the most effective use of TAXPAYER funds and for the good of all Australians. Do they really believe this? Is it possible? The article also suggests that up till now the independent schools lobby has taken a cautious approach but the gloves are coming off. Well I am telling you now Julie, that the Government schools supporters have also, up to now, taken a cautious and careful approach. They have kept quiet on the important things that implementing Gonski won’t fix, because getting a fairer funding base that is simple, transparent and adjusted according to need is so important.
But even if, by a miracle, we end up with the principles outlined in Gonski applied to education across Australia, we will still have one of the world most segregated and unequal funding systems in the world. It is a system that spends less on government schools than most OECD countries and more on non Government schools than most (see below*).
*As I outlined in a previous post http://www.onlineopinion.com.au/view.asp?article=13574
“Our funding regime for Government and non-Government schools is highly irregular in global terms. Australia sits around the middle of OECD countries ranked in terms of per capita investment in schooling. But this obscures the bifurcated elements of the funding relative to other countries. Our funding to Government schools is very near the bottom, at third lowest. But our funding to the non-Government system is near the top of the list, at fourth highest.”
Indigenous education report misses the big picture
It was wonderful to see a high quality well informed response to Helen and Mark Hughes latest missive about education failure in the NT.
The Article Indigenous education report misses the big picture by Bill Fogarty from ANU National Centre for Indigenous Studies is a worthy contribution to the debate on this important wicked issue.
It is important that the highly publicized work of Hughes and Hughes be countered. Now I know that they are very committed to Indigneous education outcomes and I don’t for a minute want to imply anything less in terms of their motivation but having read their reports over the years I have been struck by the fact that they consistently get it half right.
The poor outcomes and the poor education infrastructure and services tends to be the bit that they get right but the report wonders into simplistic demagoguery when they get to saying who is to blame. And over the years this has varied. Once upon a time it was post modernism and bilingual education but they can hardly argue this now. Then it was land rights but even that has been changed in very significant ways. Now it is teacher quality, school failure and pretend jobs.
Fogarty reminds us that there is a whole field of international research on the social determinants of education failure and that we should look to this to guide the questions we bring to this problem
He reminds us that:
“the daily routine of school in remote communities takes place against depressingly high rates of unemployment, early mortality, poor health, violence, crime, substance abuse and youth self harm and suicide.
Any consideration of education in remote regions of places like the Northern Territory must recognise the relationship between levels of attainment and poverty, health, housing, access to government services, infrastructure and socio-economic status.
These factors are not excuses for poor outcomes. They combine to constitute the reality within which teachers, students and parents battle every day to raise literacy and numeracy standards.”
But he also suggests that we can do better and shines a light on some clear areas of policy failure. His pick of examples include
- the very high untreated and unidentified hearing loss issues among remote Indigenous children
- the under-spend – current and historical – in remote Indigenous schools
- the impact of inappropriate NAPLN testing in Remote Indigenous schools
- the failure of NT to apply the funds it receives from the Commonwealth Grants Commission for overcoming disadvantage for this purpose
I agree with this list. I would add
- the serious long term underinvestment in housing which has resulted in average people per 3 bedroom housing rations of over 15 in many communities – how do kids do homework in such a setting? The current investments only start to make inroads on the years of neglect. even after new houses are built there will continue to be chronic over-crowding
- the lack of enough well trained ESL teachers in remote schools – something that is being addressed
- the lack of preparedness of teachers for the reality of remote schooling – and the subsequent high turnover rates which have a devastating impact on students (this is also being addressed)
- the lack of well designed adult literacy programs; and
- the betrayal of the trust of communities across the NT when after negotiating Remote school-Community Partnership Agreements with many communities the government turned away from bilingual education – the centerpiece of many of these agreements.
One of the reasons for the historical and current under-spend in schooling is that the NT continues to fund schools on the basis of enrollment figures modified by attendance. This is in my view a form of indirect discrimination because only remote Indigenous communities have attendance rates low enough to drastically reduce the number of teachers allocated.
But a 65% attendance rate does not mean that only 65% of enrolled students attend. All students attend but very few come every day of the week (around 20-30%). The rest attend intermittently. So funding by attendance means that a teacher would end up with quite a large group of students on their class roll even if on average there were only 20 or so per day. But they still have to plan for and teach all the students in the group. Teaching students that attend irregularly is hard enough but teaching more than you would have to in a Darwin school is just plain unacceptable.
The implementation of NAPLAN tests for year 3 students in remote Indigenous communities is a tragic farce. In most of these communities the only time students hear standard Australian English spoken is in the formal classroom. By year 3 they are no most capable of being tested in English language written literacy that all the newly arrived non-English background migrants that are exempted in other states. It is about tike there was a united push to abolish this useless practice and concentrate instead on developing and assessing children’s growing English language oracy.
Fogarty argues that the solutions to this challenge will necessarily be long term but that they should include redressing the historic under-investment and “finally, demand real commitment from all levels of government for all Indigenous students”.
I agree that this must be part of the solution but I am afraid I see no evidence of any focus on this matter. In fact the historic changes to commonwealth state funding arrangements conducted under the auspices of COAG committed parties to ignoring financial input based reporting in order to focus on the main game of outcomes. But I am afraid that in the case of the NT this was a seriously flawed backward step.
Any medium to senior level public servant in the NT will tell you off the record that ‘there are no votes in Indigenous issues’ in the NT. Until this is acknowledged and addressed I am afraid we will still be having these pointless blame game arguments in 20 years or even more. And that is shameful.
What we measure and report, and what we fail to measure: topsy turvy data
In an article in The Conversation by Kate Taylor called “Is this progress? Watering down the Millennium Development Goals[1], Taylor notes that one of the problems for reporting on progress or otherwise on the very important Millennium Development Goals is the lack of reliable data to track and assess progress.
“Our assessment of progress for all of the MDG targets remains severely hampered by a lack of data. Donors have spent billions of dollars through their bilateral programs and through partnerships such as GAVI and the Global Fund to Fight AIDS, Tuberculosis and Malaria, but they have barely invested in systems to monitor their effect.”
She also quoted Dr Seth Berkly from the GAVI alliance who noted recently that:
“ … the household data that GAVI uses – essentially the same sources used to track MDG 7 – can be very problematic. For example, in some conflict-ridden countries national surveillance is so weak that you can’t tell if 86% of children are immunised as claimed by the World Health Organisation (WHO) and UNICEF or if the true figure is closer to the 36% estimated by other experts. These numbers matter because they translate into lives saved or lost.”
Targets are important. They fundamentally underpin transparency and accountability.
This is appalling, no question and it made me think about all the other information that we do not collect or report on that are important in terms of public accountability. I can think of many examples but here are a few that matter to me:
1. Overcrowding Housing in NT remote communities
Recently Minister Maklin gave an upbeat report on the number of new houses built in remote communities in the Northern Territory[2]. On reading it most readers could be forgiven for thinking – ‘problem solved’. But the data provided does not really tell us anything at all.
Its been a long time since new houses were built in all but a few NT remote communities so this amount of new houses has to compensate for al the non developments over the past years and the population increase before it even begins to affect the appalling overcrowding situation. But none of this data is available.
When 30 new houses were built for the Wadeye community in 2007-2008, John Taylor from the CAEPR Institute at ANU remarked that over that year alone, there were over 200 new babies born in Wadeye and the average person per three bedroom home was over 15. Even if 100 people died over that same period (unlikely) it is clear that the rate of building over this period barely kept pace with the population increase. But it was hailed as amazing progress.
So to understand the impact of the new houses we would really need to know what was the average per person per home or per bedroom at the start and has it been reduced. It would also be nice to have a target. After-all if it had gone down from 15 per three bed home to 14.5 we should be stating that we have made a start on addressing the overcrowding problem in NT remote communities but we have a long way to go.
2. NAPLAN data
Unlike many I support the concept of public transparency and accountability. By this I mean that where significant funds are expended (and education is one significant area of government expenditure) the public are entitled to know that the funds expended are delivering in policy terms efficiently and effectively.
So NAPLAN data could help us – the public – understand whether we are:
- reducing the proportion of children who are failing to reach minimum standards of literacy
- reducing the learning gap that is based solely on the levels of disadvantage of students
- ensuring that children in different jurisdictions or sectors or geo-locations are not being disadvantaged by these factors
- whether a particular system level literacy intervention or major policy change is having any impact on the pattern of student learning outcomes.
Although NAPLAN is a limited multiple choice test on a narrow range of learning, it is a reasonable enough tool to use for these population level review purposes.
But this means having the data at this population level. Now we collect this data and it is published. It is included in the COAG Reform Councils Report of the National Education Agreement[3] and I am pretty sure that the information in this report has not been read and adsorbed by more than a handful of people.
This data tells us that we are not on track to halve the gap in Indigenous literacy and numeracy achievement although there are gains in some state and in some subjects. It also tells us that the percentage of children who do not achieve the minimum literacy and numeracy benchmark is up to 5 to 8 times higher in the bottom quintile of SES in each nationally and the states results vary even more. However when you look behind the data one can find enormous problems. This includes the decision to use parent education data as the proxy for disadvantage even though it is far from complete (with over 40% of NT information missing) and that student absences are so high in some states as to render the results meaningless.
But what do we do with the data – we report it at school level. Even though no-one credible has yet been willing to state unequivocally that there is any decent level of reliability or validity in NAPLAN data reporting at this level.
This school level data doesn’t tell us anything about whether we are making progress in addressing education disadvantage or closing the Indigenous student learning gap and it cant be rolled up to do this.
So there you have it. We collect poor data and barely draw on it at the population level, even though this is the most important level. But we spend all our energy, time and resources breaking it down to school level where errors of many kinds render it totally meaningless.
We have a way to go before we are data smart. Perhaps the national Curriculum can train all future citizens not to be fooled by such appalling misuse of data through better data literacy. One can only hope.
School Autonomy – IT REALLY IS ALL ABOUT THE MONEY
Ha! So the hero of school autonomy in NSW, Mark McConville, Principal of Toronto High School has recanted Principal backtracks over power shift plans – Metro & Regional – National – Education – Stock & Land..
Many of us watched the ABC Four Corners program, hosted by Kerry O’Brien, Revolution in the Classroom, where school autonomy was touted as the latest magic bullet. McConville was used as the poster boy for this solution.
What I most remember from watching this program was McConville’s boast that, thanks to the NSW School autonomy trial, he was able to ‘clean out’ his top leadership team and bring in a totally new team. No-one asked about the old team:
– why did he need to move them?
– had he tried to create a leadership team out of them?
– where did they end up?
– how did this event impact on their morale or the morale of the schools they ended up at.
Now he has changed his tune. He has realized that this is a process of massive cost shifting, as the work now done by the Department of recruiting and appointing teachers is devolved to schools, along with the responsibility to manage budgets – possibly shrinking budgets
Back then he said that devolution gave him a chance to drive big reforms at his school – reforms that are kicking goals. Now he is saying that “the benefits for the school from the trials had been largely financial’.
Now he is saying
We don’t want to be saddled with the staffing budget, with the potential for cost-shifting and cost-cutting [from the department to schools]. And we don’t want to go from making educational decisions to making financial decisions.
All perfectly reasonable which is of course why there is industrial action.
He also supports claims that have been made by many who have reviewed school autonomy in Victoria and elsewhere. He admits that he would be hard-pressed to cite anything a school could do under the local authority reforms that it wasn’t already able to do, to a certain extent, already and that the only area where savings can be made is in staffing.
If any NSW principals were sitting on the fence on this matter, this view from the voice of experience should make them sit up and take notice.
