Unpicking the student attendance/learning relationship in remote Indigenous schools

An ABC News report today sparked my interest with the headline Study finds no relationship between attendance and results at remote Indigenous schools.”

This interim finding comes out of an important five-year study, The Remote Education Systems Project that is looking at how education can best meet the needs of remote communities.

Specifically researchers have concluded that increasing attendance at remote Indigenous schools will not necessarily improve results, because, according to a senior research fellow, Sam Osborne, the analysis of the NAPLAN results from more than 200 remote schools has not identified any established relationship between attendance and outcomes in NAPLAN results.  This puzzling finding is the focus of this post, but first the research project itself is worthy of comment.

The Remote Education Systems Project

I have just come across this study. It looks very exciting and promising for a number of reasons.

Firstly, it is over a number of years – its final report is not expected to be completed until the end of 2016.  This is unusual in the world of Indigenous research.  Most research attempts to measure the impact of a single initiative, sometimes trying to use random trial type approaches, as though the drug trial paradigm can apply to such a complex context.

Secondly, it is focussing on remote Indigenous schooling only.  I have long argued that the tendency to assume all Indigenous education disadvantage challenges are the same, regardless of the context, has been most unhelpful.

Thirdly, the research is not starting out with a standard deprivation model.  It takes as its starting point the importance of community perspectives and understandings, and of building on community strengths.  I am really looking forward to the insights that will emerge from fresh thinking and community engagement.

And finally, the research framework is ‘going outside the box’ and questioning whether the current outcomes/targets/standards based approach is the best way to go.  I have recently made the case for the irrelevance of all the NIRA targets and output measures related to schools in the remote Indigenous context, so I strongly support this.

However, in questioning whether the dominant standards/target based approach as the only way to go, I do hope that the researchers also remember to take a critical look at the data and not pass over what else it might be telling us apart from its lack of fit.

The attendance/student learning puzzle

The issue of school attendance is an excellent case in point.  Systems prioritise attendance because of the assumption that children cannot benefit from school unless they attend on a regular basis. This makes intuitive sense.  So if the connection between attendance and learning progress is not obvious in remote schools, what else might be going on?

Now I don’t know how this project is using the attendance data but most educators who write on this matter argue that children need to attend school for over 80 per cent of the time to benefit.  Others insist that they need to attend for over 90 per cent of the time.  However this is all rather irrelevant to the performance data available to most of us, because attendance data is rarely reported to the individual student level.  It is only reported to the school or classroom level so comparing high attending individual students with low attending individual students is not generally possible.

If this project has managed to analyse results by individual student attendance profiles this would be an important piece of work.

However, whether or not this is what has been achieved by the study,  understanding how it is usually reported and how this is used  in policy terms is important.  Let me explain

The MySchool and the COAG reports both use the concept of average attendance.  This might be ‘good enough’ in the majority of situations where average attendance rarely falls below 90 per cent. But in remote communities, where average attendance rates of 52 per cent are common, it is not very helpful.

Average attendance basically measures the collective gap between the number of students enrolled and the cumulative number of students on a daily basis.  So if there are 200 days in the school year and 30 students enrolled, 100 per cent attendance would mean that over the school year there were 200×30 or 6000 ‘student attendance events’.

However, a 52 per cent attendance rate could mean anything from, 52 per cent of students enrolled attending for 100 per cent of the time, to all students attending for 52 per cent of the time, to anything in between.  In the first extreme, 52 per cent of students should be benefitting but 48per cent of children would get absolutely zero benefit.  However in the second extreme scenario, none of the children would benefit.

Of course, most schools are not at either extreme but we still don’t know what an average attendance figure means in terms of its impact on student learning.  When I last had access to the NT official and very detailed attendance data base (over 3 years ago), I do recall that for the larger remote community schools, the average number of students who attended school over 80 per cent of the time was only 27 per cent.  Yet the average attendance rate was around 60 per cent.

If we delve further into what this might mean at the school and classroom level, some new questions emerge.

So, let us imagine a classroom in a remote Indigenous community school with a 60 per cent attendance rate where 27 per cent of the children attend over 80 per cent of the time.

Firstly, how many children would be on the roll for the average class if the official teacher-student ratio is 1-20?

In the NT, schools are allocated staff based, not on enrolment numbers, but on attendance[1] . This impacts significantly on actual classroom size and the challenges facing remote teachers. For example, a primary school with 300 children enrolled ,but an attendance rate of 60per cent, would be allocated staff for 180 students[2] not 300. Yet the number of students who need to be assigned to teachers and classes is 300 not 180 – they just attend irregularly. This would require making class sizes of about 33 not 20.

So on any one day, a teacher might have only 20 children in their class but about 33 children on the roll.  Based on the expectation that only about 27 per cent would attend over 80 per cent of the time, this class of 33 might have about 9 children who attend on a very regular basis and the remaining 24 children would also attend, albeit on a highly irregular basis.

Can you just imagine the chaos of such a classroom and how hard it would be to focus on the small number of students who are there regularly?  Add to this mix, inexperienced short term principals, a high number of novice teachers, a generally non-English speaking student body and cultural challenges, and you get an even more accurate picture.

It appears that the finding, that the rate of attendance does not necessarily lead to improved learning outcomes, may well be because of discriminatory school funding by the NT government and the impact of the high level of irregular attendance of the majority of students on the classroom learning environment.

Knowing whether this is the case matters because it affects the questions we ask and the solutions that are considered.

I do hope that this issue is investigated further as part of this exciting research project.

[1] This, in my view, is a very serious case of indirect discrimination.  It is also highly unethical because the NT government signed a Memorandum of Understandings with the Commonwealth Government in September 2007 that included a commitment, on their part, to move from “staffing based on attendance” to “staffing based on an Agreed Student Number” (note: this would, be based on estimates of the numbers of age relevant children in the designated area, so it would expected to be, at least at the level of enrolment, but possibly higher). This work has never been done. This has enabled  the NT Government to continue to underfund Australia’s most needy schools for years.  One of the reasons they can get away with this is the COAG approach of only requiring output based accountability.  It is also worth noting that, had the recommendations of the Gonski Report been implemented, there would have been an independent monitoring body to monitor needs based funding.

[2] The savvy reader may have noticed that the MySchool data on FTE student numbers and FTE teacher numbers in NT remote schools does not bear this out. In fact these ratios look very healthy.  I have hesitated writing about this issue because of this problem.  But I now understand how to make sense of it.

In the explanatory notes of the Productivity Commission’s Report on Government Services  (ROGS) Chapter 4 on Schools, the following note is included under the definition of teacher: For the Northern Territory, Assistant Teachers in Homeland Learning Centres and community school are included as teaching staff. ( 2013 p 4.9.9).  This labeling of unqualified Indigenous Education Workers as teachers is another NT sleight of hand. For instance, it allows them to create an impression that the Homeland Learning Centres have daily access to a teacher, but they do not.


The Incredible CREDO: claims that its charter school research verging on criminal

I am posting this critique of the CREDO, because my previous post talks about the CREDO research on the comparative performance of Charter schools relative to public schools in the US.  When writing the article I had not read this report by Jason France, a former Louisianna Department of Education  employee  CREDO is not credible, and never has been | Crazy Crawfish’s Blog.

It is clear that underneath the surface where administrators, researchers and organisations work to produce evidence relating to education policy, there  exists a shadow world where people’s official position is less important than their political connections and the politics being played.

It seems the CREDO research suffers from this.  This posts conclusion is that

CREDO is simply not credible, they are not a research institution, they are pro-charter propaganda churner and should be classified as such by anytime anything they produce is quoted in an newspaper or news program that claims to be unbiased and impartial. If you are a parent, please do not pay CREDO any more attention than you would a miscellaneous propaganda pamphlet handed out at neighborhood grocery store, or stuffed under you front door handle. You can see CREDO as a joke, a wolf in sheep’s clothing, a PR firm or a charter school pimp, but an independent research organization they are not.

If Independent Government Schools are the answer: what is the question?

Pyne believes that introducing Independent public schools across Australia will bring significant benefits to these schools and their communities.

Yesterday The Conversation published its fact checker that concluded that the claims to increased productivity and efficiency as well as increased student outcomes have no basis in evidence.

While I agree with this, I think this was a cautious assessment that drew its areas for consideration too narrowly. In this post I focus on some of the more concerning aspects about the IPS system that were not considered by the fact checker.

Claims of improved student outcomes – treatment of the research

But first a brief comments on the claims that were considered.  The fact checker, in looking at overseas evidence of schooling set ups that have similarities to IP schools, looked at Charter schools in the US.   It drew from the 2013 CREDO Charter Schools study of the comparative student learning effects of Charter Schools.  This report concluded that there were some comparative learning outcomes improvements but that they were non-significant in nature.  Most media headlines reported in terms of Charters are performing slightly better than public schools

What I find interesting about this is how this non-significant difference is treated.   The Great Lakes Centre for Research recent Review of the CRDEO study makes this point

 The most important results of the study…are differences of 0.01 or 0.02 standard deviation units, and even the largest effect size reported are on the order of 0.07 standard deviations. 

Hanushek has described an effect size of 0.20 standard deviation for Tennessee’s class size reform as ‘relatively small’ considering the nature of the intervention.

So there you have it – an effect size that is tiny- very tiny – is hailed as a small improvement justifying this large scale reform.  However it is much smaller that the effect size attributed to smaller class sizes by Hanushek, who led the campaign to oppose class sizes because, the effect size is too small.

The logic behind autonomous schools

To go beyond the fact checker scope it is necessary to dig behind the claims.  Pyne is arguing that Australia has invested strongly into non-Government education and it is working well for Australia.  He notes that we are unusual in our high levels of investment in non-government schools relative to other OECD countries – so we must see it as a public good.

He is asking the reader to assume that non-Government school enrolments skyrocketed over the years of the Howard regime just because it was a great idea – totally demand driven. But I won’t chase down this particular rabbit hole here.

So, says Pyne, we have these great institutions that work well, so lets get a piece of this into the public education system.  This implies without any evidence that the public system is not working so well.

So what he is borrowing from the non-Government system?  Is it the great facilities, or the ability to enrol students as they see fit, or their ability to charge fees or their superior levels of per pupil funding.  No – because non-Government schools can only selectively enrol students and charge fees because there are government schools that must then pick up all the non selected students, and provide a free education

What he is picking up, is the stand-alone school concept, minus the generous funding – a school with a bucket of money to do its business, responsible to a board and able to make all its own decisions. This will, he argues, be more efficient, will encourage bold new thinking and innovation, and will give the community much more say over spending priorities.

It is interesting to note that in the negotiations over Gonski the non-Government sectors successfully argued for additional systemic funding to better support their stand alone idealised schools. Maybe, just maybe, stand-alone models are not all they claim to be.

The previous WA Education Minister, Barnett justified the WA model of IP schools in terms of increasing competition and variety because maintaining all schools as equal was undesirable as it breeds mediocrity.

So to follow the logic pathway, IP schools will deliver better student outcomes, more productivity and efficiency because ‘stand alone schools’ will make all their own decisions about how they use their bucket of funds.  This will make them more competitive, they will spend the same amount of money more wisely and they will be more innovative.

So lets look at these claims

IP Schools will be more innovative

A WA press article recently profiled an IP school in WA that opted to become a marine biology school.  Fabulous example!  This school has reported that student engagement is high and that they have a big enrollment waiting list.  Students in its enrollment district have an automatic right of entry but students outside this district will have to move house or hope for an enrollment win.

However, NSW, arguably the most centralist state when it comes to its schooling has schools that specialize in agriculture, in performing arts, in technology, in sports, in languages.  There are schools with Opportunity Classes and the Board of Studies has a year 12 syllabus in Marine Studies.  Victoria has Government schools that offer Steiner programs, ACT has the Cooperative school and a bilingual French-Australian K-12 IB school.  There are networks of schools that adopt innovative approaches such as the Big Picture schools, IB schools, UN schools, Stronger Smarter school leaders and Dare to Lead schools. These are just a few examples I know about.  We don’t NEED IP schools to develop innovative schools within the government system.

Lyndsay Connors, argues that when she was involved with the National Schools Network – an initiative of the Hawke-Keating government intended to free schools from bureaucratic and union rules, the new and innovative practices that schools adopted, that she witnessed, were all ones that they could have done without special freedom.

She says this was also true of the self-governing schools created within the Victorian public system under the Kennett government. A few principals took the opportunity to create a governing school or board with some financial freedom, such as increasing salaries, but she says other innovations she knew of depended on extra funding.

Connors argues that with the same increase in funding, other schools could have implemented similar reforms, even while operating under a more centralised system.

And of course not all innovations are good innovations.  A school could decide that they could shift funds directed to ESL learners or special needs students to a program that the more influential members of the board might want – a violin program, or an artist in residence.  Having parents on boards does not always lead to decision that are in the best interests of all parents.  Articulate ‘entitled’ parents will always end up with more say.

And finally, lets remember that some of the innovations in Charter schools are very worrying – “no excuses” schools that feed the school-to-prison pipeline, or, schools with built-in churn as they rely almost solely on TFA teachers passing through education, en-route to a high profile future.

Pyne is allocating $70 million to this initiative.  This will give all participating schools about $47,000 as a one off allocation. So all the innovations will have to come from changing the staff profile in some way because that is where the vast bulk of the funds are spent.

 Competition improves schools

Proponents of this view argue that by giving parents the power to choose between schools and the power to influence schools, schools will work harder to earn more student enrollments.  This competition will improve all schools.

It is very clear that in WA, where only some schools are IP schools, this competition has been hard for non-IP schools.  Trevor Cobbold has posted extracts from principals about the effects of the IP arrangements on their work.  They talk about how the IP schools suck up all the highest rated teachers, while they are forced to staff based on redeployees.  And the more high needs the school, the more intense the problem.

Here are some of their comments:

 Basically, the better ranked teachers chose better schools. That is how it goes and that is how we get residualisation within schools. Low SES schools just cannot compete with the leafy greens, and they don’t even have to be leafy greens but good solid communities that support education and their kids in school. There was always a component of this, but IPS has really amplified it.


[this is not a low SES school]
Public education was once about equity, about being able to say that a child way up in Wyndham and a child at leafy Wembly Downs will get the same quality of teacher. Creating a privileged set of schools badly damages this concept.

Autonomy and Student Equity

The ACER evaluation of the impact of IP schools in Australia did not ask questions that might have exposed the impact of this set up on student equity.  They did not look at any changes in the enrollment share of IP and non-IP schools by student demographic characteristics, nor did they look at the changes to the staffing profiles of the schools.  In my view this is a pretty big omission – not necessarily of  ACER’s choosing.

This is the big issue with Pyne’s proposal in my view.  Trevor Cobbold makes this point

 Greater demand for IP schools amongst higher income families and increased flexibility of IP schools to select student enrolments is likely to lead to more social segregation between government schools in WA. Inevitably, it will mean increased differences in school results and more inequity. This is after all what a market in education is designed to do.

 Chris Bonner reiterates

But the bigger danger is that we risk losing the equity safeguards which our public school system, with all its claimed faults, currently provides. [Where schools can choose their own teachers] … the best will gravitate to the schools with the more valued location, easier to teach students and money.   ….there are no prizes for guessing which schools and communities will miss out.

 There are other hidden stings. Unless closely monitored, increasingly autonomous public schools will seek and gain greater control over student enrolments. I love them dearly but already there are few rules which get between many of our enterprising school principals and a desirable enrolment. The better placed autonomous public schools will join their private counterparts in applying both overt and covert enrolment discriminators, worsening the complex equity problems revealed by the Gonski review.

A blog post by Chris Lubienski about research into schools autonomy and equity in the New Zealand context gives us a glimpse into how enrolment manipulation is likely to happen over time if autonomous schools are introduced across the nation. He found that schools will actively pursuer policies of enrolment segregation if they are given a chance to do so and that autonomy initiatives provide just that sort of opportunity.  His findings are so important I am quoting from him at length:

Previous research has shown that schools in more affluent areas are more likely to be in greater demand, and thus more likely to have enrolment schemes.  The question we asked was whether these self-managing schools were using their autonomy to draw their zones in order to improve or restrict access for disadvantaged students.  To do this, we simply compared the level of affluence in a walkable radius around each school to the level of affluence in the boundaries that the schools themselves had drawn.  Certainly, school zones are not perfects circles, as their creators have to consider traffic patterns, geographic barriers, and the boundaries of competitors.  But, all things being equal, we could expect that deviations in those boundaries from a geometric radius around a school would be more or less equally likely to include or exclude more affluent neighborhoods.  

But that is not what we found.  Instead, there is evidence of rampant gerrymandering to exclude children from more disadvantaged neighborhoods.  In the cases where there is a statistically significant difference in the “deprivation level” of the population in a school’s drawn zone compared to its immediate area, over three-quarters of these self-managing school had drawn a zone that was significantly more affluent than their immediate vicinity. 

Moreover, as if to add insult to injury, more affluent schools are not only drawing boundaries to keep poor kids out, but in their promotional materials are bragging about their success in doing this.  A review of school websites shows that more affluent schools are much more likely to include official information about the number of disadvantaged students they serve. 

While we might find these types of practices to be distasteful for public schools that are funded by taxpayers to serve all students, in some ways, such actions are predictable (if indefensible).  After all, policymakers are creating education markets where schools recognize competitive incentives to shape their enrollments.  It should be no surprise that, given such autonomy and such incentives, they find creative ways to do just that. 

So if on Saturday we have a change of Government, this is what we can look forward to in our schools.  We will have a tiered system of schools, competing on a highly unequal basis and our already highly segregated education system will become even more so.

Ironically one of the best tools for highlighting the issues will be the data from MySchool.  Is this why Pyne thinks the publication of the NAPLAN results is a bad idea?  It sounds crazy but I do wonder.