Rejoinder after ACARA response: The silent tragedy of NAPLAN, students reported in misleading bands

First I would like to thank ACARA for engaging with my blog The silent tragedy of NAPLAN, students reported in misleading bands.  It is great to see that we still have an attentive and responsive public service. For that I am grateful. My problem is with the Australian body politic that drives some of NAPLAN’s policies.

First a quick thanks to all those who responded to the blog. The response was heartening.

The point that I was trying to make with the blog was that there are random students doing NAPLAN who are getting widely misleading results.  Probably not a high percentage, but when over a million students undertake the test, 5% is 50,000 students. Most of those students might not even be psychologically affected. But some, perhaps those who tried hard on the promise of an ice-cream, or those who tried hard to please mum who is going through a rough trot, or those who tried hard one last time to be good at numbers or words, will be. Out of one million, the number of students may be less than 10,000, or less than 1%.  But this worries the caring teacher type as it causes unnecessary grief.

Students, perhaps more than adults, are particularly vulnerable when things are unfair. Students roughly know where they are with their school work. When they receive feedback that is fair, justified and agrees with their self-perceptions, they generally accept it thoughtfully.  Fairness is a big thing in testing (for example, see Camilli, 2006; Zieky, 2015).  When feedback is unfair, students can have maladaptive emotions (for example see Vogl & Pekrun, 2016).

As a former maths teacher, I like the mathematics of assessment, often finding it more beautiful than useful. I like the simplicity, elegance and flexibility of the Rasch model (Adams & Wu, 2007; Rasch, 1960). I like the magic of plausible values, that random numbers can sometimes be more useful than real ones (Wu & Adams, 2002). And what is there not to like about a number called a warm estimate (or WLE), that gives a student who doesn’t get anything right a non zero score.  But this magic does not always work for me with NAPLAN reports.

Response 1 (it gets a bit boring here)  

 

Link to ACARA NAPLAN equivalence tables

Some points

  • in the blog, I mentioned that confidence intervals were not included, I didn’t mention standard errors (trivial difference though)
  • The confidence intervals are not shown on student reports (from what I’m told)
  • the equivalence tables for 2016 is 26 pages of raw numbers, hardly amenable for parents.

A hypothetical example, using the 2016 Year 3 Spelling Score Equivalence Table, (see below)

A student gets a raw score of      13

from table, is a scaled score of      439.7

from table is reported band      5

from table scale Standard Error      20.36

therefore 90% Confidence interval = 1.64 x 20.36        33.4

therefore 90% Confidence range          473 (Band 5) – 406 (Band 4)

so this student’s graphic report looks something like this (on top) then annotated below

 

So my claim that student scores are not reliably reported in NAPLAN remains, as does the observation that this unreliability is not clearly communicated to students and parents. Further, it used inappropriately by the media to label improvers, coasters and strugglers .

Response 2

Sure, and most school-based paper-and-pencil tests administered by teachers are administered under classical test theory (CTT) principles. Perhaps CTT tests lack precision, but student scores relate to the content on the test which enables teachers to give meaningful feedback around that content on the test.

NAPLAN, is based on the Rasch (1960) principle that the raw score is the sufficient statistic and reports on levels which is abstracted from the content (read the book). This makes it close to impossible to give meaningful feedback. Teachers would need to administer another test, or get evidence from elsewhere,  to provide meaningful feedback.

Online may provide enhanced precision, but the feedback will remain less useful, and that students within cohorts will do different forms, teachers will need to trawl through test forms to have any hope of providing feedback. Further, the socialising narrative of the test will be diminished, and likely to alienate marginal students.

 Conclusion

Moss (2003) argues that in “much of what I do, I have no need to draw and warrant fixed interpretations of students’ capabilities; rather, it is my job to help them make those interpretations obsolete.”  The reporting regime of NAPLAN makes results, in the terms of Austin (1962), performative. That is, NAPLAN reporting does not so much describe, but they create and define.    Performative statements are not true or false, but either happy or unhappy.  Some of NAPLAN student reporting is unhappy.

(I’m happy to amend or withdraw if there are errors, let me know)

 

 

Adams, R. J., & Wu, M. (2007). The mixed-coefficients multinomial logit model: A generalized form of the Rasch model. In M. v. Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models (pp. 57-75). New York, NY: Springer New York.

Austin, J. L. (1962). How to Do Things with Words. Oxford: Oxford University Press.

Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221-256). Westport, CT: American Council on Education and Praeger Publishers.

Moss, P. A. (2003). Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and Practice, 22(4), 13-25.

Rasch, G. (1980). Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: MESA PRESS.  (Original work published 1960)

Vogl, E., & Pekrun, R. (2016). Emotions that matter to achievement. In G. T. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 111-128). New York: Routledge.

Wu, M., & Adams, R. J. (2002). Plausible Values–why they are important. Paper presented at the International Objective Measurement Workshop, New Orleans.

Zieky, M. J. (2015). Developing fair tests. In S. Lane, M. R. Raymond, & T. M. Haladyna (Eds.), The handbook of test development (2 ed., pp. 81-99). New York: Routledge.

The silent tragedy of NAPLAN, students reported in misleading bands

The release of international reports on education as well as NAPLAN have placed teachers under much pressure. Most of this pressure arises from innuendo, or what statisticians call correlations. Is this pressure warranted?

NAPLAN reporting of student abilities is unreliable. This is likely to have tragic effects for some students. These individual tragedies are largely silent except to the student. The dubious accuracy of NAPLAN results questions the fairness of recent media reports that label students as big improvers, coasters, and strugglers.

NAPLAN reports student results as dots within bands numbered from 1 to 10. That these dots are solid conveys a sense of certainty, a certainty not matched by the mathematics. It is normal practice in statistics to show a confidence interval.  For example, a 90% confidence interval would show a range in which we are 90% confident a student’s ability is located.  NAPLAN does not report these confidence intervals for individual students.

Margaret Wu (2016) finds that if NAPLAN included confidence intervals, it would not be possible to confidently locate a student in a particular band. That is, around one in ten students is being reported in the wrong band. This effect is random and potentially has tragic consequences.

Over one million students do the NAPLAN tests so there are over one million stories. Once the unreliability is considered new stories emerge for our improvers, coasters and strugglers. Improvers, for example, could simply be those students reported below their level one year, and above their level the next. Most students would be coasters. In statistics, this is regression to the mean.

While most students would receive a NAPLAN score close to where they should, about 10%, or more than 100,000 students, receive a misleading message. This includes students who may have tried hard to improve, only to be randomly reported below their real level. It also includes students who are coasting, but are randomly reported as excelling. Both types of misleading messages affect student motivation. That these little tragedies are occurring in large numbers is likely to be undermining Australia’s international performance.

NAPLAN doesn’t assess curriculum, it only  “broadly reflect aspects of literacy and numeracy within the curriculum in all jurisdictions” (ACARA, 2016). If teachers were to teach only ‘aspects of curriculum’, and provide student feedback in the haphazard fashion of NAPLAN, they would be ridiculed.

Teachers are being held accountable to dubious statistics. For example, the American Educational Research Association (2015) strongly cautions against the use of value-added-models. Yet Australia reports student progress without reservation or qualification on the My School website (myschool.edu.au). This is not in the interest of students, teachers, or schools. In whose interest this reporting is occurring remains opaque.

Australia’s education measurement industry is a plagued with vested interests. With over 300,000 Australian teachers, everybody wants a piece of the pie. Teacher training, teacher supply, and teacher development provide commercial opportunities. This feeding frenzy is a disgrace and should stop.

addendum: Link to my rejoinder to ACARA’s Twitter response

ACARA. (2016). NAPLAN Achievement in reading, writing, language conventions and numeracy: National report for 2016. ACARA: Sydney.

American Educational Research Association. (2015). AERA Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs [Press release]

Wu, M. (2016). What national testing data can tell us. In B. Lingard, G. Thompson, & S. Sellar (Eds.), National testing in schools: An Australian assessment. London: Routledge.

PISA is about education systems, not teachers

Education can be talked about in two ways, as a culturally lived experience, or as a commodity. It’s like marriage, you can talk about the depth of the relationship, or talk about the size of the ring. As we begin to digest the latest round of international reports on educational achievement we have a similar choice. We can discuss the lived experience of students and teachers, or simply discuss the numbers. This article focuses on the culturally lived experience of education.

Assessment is universal in education. It can be as simple as a teacher asking a student “are you ok?”, or as complex as an international assessment program. Both show a concern for students and demand a response. Teacher assessment demands a response from teachers, and international assessment demands a response from systems.

Those that consider education a commodity will look for that golden nugget, that vital statistic that explains, and brings kudos to the finder. Those that consider education a complex cultural process will know that these statistics are a faint echo of a multiplicity of intersecting systems. They see the diversity of interests that go into making a school, a classroom, and learning happen.

Seeing children struggle can be painful, particularly for teachers. Some will seek to relieve feelings of guilt by conjuring a LouAnne Johnson from Dangerous Minds. These silver bullet solutions are consistent with our own educational experience –  where one good teacher changes our lives.  Some gladly proffer a LouAnne Johnson, for a fee. Others recognise that these initiatives are superficial with no system impact. It only leads to marketing and publicity that demeans the broader teaching workforce.

Education systems are complex.  There are legislative complexities, with different legislative frameworks for the early years and for the school years. There are jurisdictional complexities between state and federal governments. There are many systems for managing teachers and managing finance. There are also complex information systems, building and infrastructure systems, payroll systems, curriculum systems, assessment systems, and various support systems. Each with its own jurisdictional and legislative complexity. Each with its own problems and levels of both competence and incompetence. These systems cater for over 3 million primary and secondary students across Australia.

Most in education work hard and are dedicated to doing their best. Particularly teachers, who bear the sins of systems, the sins of students, and the sins of colleagues. Teaching is hard, its emotionally draining, tiring, and often quite isolating.  Opportunities for teacher professional development are sparse compared to corporate careers, making diffusion of innovation difficult. Yet teachers are exposed to high levels of control, monitoring, and auditing.  Nevertheless, most teachers work in good faith and embrace systems, explain them when required, and apologise for them when they fail. They are primed for the sucker punch.

The sucker punch comes every couple of years when international test results such as PISA, PIRLS and TIMSS are released. Teachers react to such media storms as they would a bushfire that has burnt their school.  Teachers first internalise the bad news, then frame hope, then look for lessons to be learned. This makes them prone to attacks from commentators, researchers and think tanks. Teachers react in good faith by listening for lessons, and their instinct to defend education is deliberately misconstrued by some as recalcitrance. Teachers are not to blame, systems are.  Teachers are vulnerable.

Some believe that over two hundred thousand teachers can, and are, conspiring in an ideological plot. Various quick fix solutions are offered. Perhaps an ideological fix, a silver bullet teaching approach, a handful of glamour young wiz bang teachers, or a career restructure. All these suggestions fail to take into account the systemic nature of education. These suggestions are in the main glib, and cast teachers or teaching as the problem, they demean the profession and are hurtful towards those in it.

Suggestions are often portrayed as ‘strategic’, but this is misleading as they are mostly bereft of strategy and better characterised as opportunistic.  They often use hard data such as PISA, TIMSS, PIRLS and NAPLAN, but lack the sort of finesse encouraged by strategic thinkers like Henry Mintzberg. Instead they just rehash data and add spin, often with a vested interest. They often proffer simple quick fix solutions from overseas, in doing so they ignore the strategic work of Michael Porter, who argues that in order for nations to be competitive they must develop capacity and appropriate industry structure.  Those offering solutions from overseas have greater interest in arbitrage than education.  Then there are those that ignore Mark H. Moore’s work on authorising environments, and fail to take into account the legislative and jurisdictional complexities of the quick fixes they propose. In being blind to the systemic nature of education, they only see the teacher, and blame them. Then there are those proposing mindless assessment, without regard for system complexities, as shown by delays in getting NAPLAN online. These suggestions by commentators, think tanks and researchers are devoid of rigour.

Systems need to respond to the latest round of testing results, not teachers. This necessarily requires structured engagement from intersectional interests across the system. Australia could revisit outcome focused education.  Outcomes focused education is when abstract educational measures are used as economic proxies in a neoclassical economic supply and demand sense. Australia has been doing this for a number of years, using data from programs such as NAPLAN, PISA and TIMSS to control the work of teachers through markets.  This has been at the expense of a concern with educational inputs such a syllabus, curriculum, teaching practice and so forth. There is much assessment for monitoring, but not much assessment for providing useful information to teachers. The neoclassical economic approach to education appears not to be working.

Another possible area to look at is transparency of vested interest.  While there are good arguments for competition for services such cleaning contracts, the argument for competition in education is less compelling. Nevertheless, the rise of markets and technology within a relatively buoyant economy creates an educational environment with intense competition. Particularly for services such as testing.  It is often difficult to decipher if calls for more testing are motivated by educational interest or by vested financial interest. Particularly given the significant amount of cross organisational board membership and financial interests in the sector.  These interests span government, not-for-profit, and private sector organisations as well as universities. The lack of transparency could be draining the energy out of the education system and be causing Australia’s educational decline. It’s an area worthy of inquiry at the governmental level.

Given the expenditure on education, economics is an important consideration. David Gonski’s report on funding is a good example of how PISA and NAPLAN data provide a useful broad brush for identifying issues. The recent Productivity Commission report had similar conclusions and draft recommendations relating to data. But it was disappointing that the Commission did not make a greater effort to look at the sector and industry structure. There is little depth to these analyses from an industry perspective.

Teacher education is another area rife with issues, not because of the people working in it as often portrayed by simplistic commentators. Teacher education has complex jurisdictional issues in a climate of higher education reform and the rise of markets. Teacher education should be an arena of intense cooperation, but the rise of markets often means Deans of Education need to compete with each other. Academics too are often torn between servicing the broad needs of teachers, and servicing the specialised needs of academic publishing for career progression. Educational leaders could be addressing this issue, but ignore it perhaps due to vested interest. Particularly as private players are seeking entry to this potentially lucrative market, in often opaque ways.

The obsession with teacher and teaching quality is puzzling. It demeans the individual for faults of the system. Rather than teacher quality, there are bigger issues with many teachers teaching out-of-field through no fault of their own, particularly in mathematics and science. There are also structural issues with teachers on short term contracts. These are industry wide structural issues, not the individual teacher, or teacher education, issues.

Education is a complex system for which teachers and teacher educators are the public face. However, international testing and reporting do not address these vulnerable teachers. Teachers will naturally be curious and gain much from reading the reports. However, the purpose of these reports is to hold the system to account and to inform policy development and reform.

terpodcast on which this blog is based

What is outcome focused education?

Outcome focused education is not so much an approach or philosophy. It is remarkable for what it lacks and is akin to Adam Smith’s ‘invisible hand’.  It has two components, neoclassical economics and Rasch-based measures. It lacks a focus on curriculum.

The knowledge and skills valued by society are traditionally determined by policy-makers in consultation with stakeholders who implement systems to develop and deliver curricula consistent with community needs, expectations and values. Teachers, relevant organisations, and government authorities work together to generate shared meanings that are expressed through curriculum. However, it is these activities that are increasingly missing from an outcome focused approach to education. This void becomes manifest when educational issues are identified and solutions are proffered from overseas.

Seminal developments in both the Rasch Model and neoclassical economics come out of Chicago. While George Rasch was originally from Denmark, it was at the University of Chicago that researchers such as George Wright and Australia’s Geoff Masters made seminal developments during the 1980s.  The Rasch model is mathematically elegant, and perhaps even beautiful in its simplicity and generalisability. It is a model that informs assessment measures using link items with wide horizons. PISA measures, for example, have horizons over participating countries and over time. Similarly, NAPLAN measures have horizons over Australian states and over time. For trend reporting, these horizons are necessarily anchored, and for PISA and NAPLAN they are anchored on conceptions of knowledge defined in the initial cycles of PISA. So while the measures were good at the time, and remain very relevant, they haven’t evolved much in consideration with things such as the internet and graphic calculators.

While PISA initially made a big effort with its testing material, good content is not really required for using the Rasch Model. George Wright lauded the Rasch model on the basis that it transcends the questions, and transcends the measuring instrument.  So educational content is not so important for Rasch. This is reflected in NAPLAN for example, with ACARA only claiming that NAPLAN broadly reflects aspects of literacy and numeracy within the curriculum across the states and territories. So NAPLAN doesn’t actually assess curriculum, just somewhat assesses curriculum. Due to the need to report trends, NAPLAN is anchored in aspects of curriculum from several years ago.

The beauty and the pernicious of the Rasch model is exemplified in the diagram below. It illustrates two students doing a rudimentary mathematics test. Student A can do all four operations but only with small numbers, Student B can only do two operations but with both high and low numbers. Yet their scores are the same, and their Rasch measures would also be the same.

pisa

What is remarkable about this example is that based on the Rasch scores these students are the same.  That is, in more complex assessments such as NAPLAN with a set of broader tasks, similarly varying skill profiles are not able to be detected. Rasch measures are not useful for determining what a teacher should teach next. These measures are only useful at a system level. Teachers are left to scramble in the dark. They may be implored to “dig into the data”, but that well is really dry; there is no more information there. NAPLAN doesn’t provide the sort of information a teacher needs to target or differentiate teaching.

NAPLAN and PISA are useful for system management in a broad brush economic sense, they provide a means for bureaucrats, think tanks and pundits to meaningfully talk about education.  They provide a link to neoclassical economics by providing proxies for output in the supply and demand equation. They are used in reports by Gonski and the Productivity Commission. They are also used by think tanks such as the Grattan Institute, and Mitchell Institute.  It is noticeable that these reports generally lack reference to curriculum documents or pedagogy. In this way, think tank commentary is both out of date and out of context. It is meaningful in that these data are the best data available at the system level, but most commentary is vacuous discussion about numbers, not meaning. Everybody feels like an economist when they talk using PISA and NAPLAN data, and nobody feels like a teacher.

NAPLAN and PISA are not useful for talking about students or teachers, mainly because these tests don’t reflect the work that they are engaged in. Teachers and students are engaged in curriculum, while neither NAPLAN or PISA are.

Teaching is a rare profession from a Human Resource perspective.  Their work is judged on measures that have nothing to do with their roles.  Further, any overt attempts at achieve targets is frowned upon, even called cheating.  I often wonder how long it will be before this house of cards is formally challenged in the courts.

 

Is this the end of outcome focused education?

There will be much media reporting in the next few weeks arising from the recent release of PIRLS and TIMSS, and the imminent release of PISA. Should the results be good, some will credit system management, others will credit teachers and teacher educators.  Should the result be bad, these positions will of course be reversed.  The position taken basically boils down to one of two beliefs. One belief is that 250,000+ teachers are capable and actually engaging in some sort of conspiracy, while a more rational alternative is that results are products of the system.

To date, the commentary from think tanks and peak bodies have been remarkably shallow.  They do not do the strategic work they purport to do.  They tend to blame teachers, teacher educators, or ideology, and proffer suggestions for faded educational fads from the past, such as synthetic phonics, new forms of assessment, a particular product (buy 5 Fingers Literacy now for only $9.99!).  Such commentary is both superficial and unhelpful, with teachers generally already doing their best to incorporate useful practices to provide balanced, inclusive approaches.

As an aside, among the more insidious suggestions tend to be small impact projects that generate publicity. Examples include exiting disengaged teachers at $50,000 a pop, or placing selected graduates straight into the classroom. These types of initiatives have very little system impact, but tend to do well for those managing them, and create much shade for the rest of 250,000+ teaching workforce.

Economists also seek to provide a reasoned response, but they rely on PISA and NAPLAN data. The big problem with these data is that they have reified conceptions of knowledge. This is because they rely on link items and equating techniques. Unfortunately, while economists understand things like Net Present Value, they don’t understand educational assessment and how it evolves over time and how it has context. So, for example, PISA is using conceptions of mathematics and science before the age of Facebook, twitter and YouTube . They do not reflect Australian curriculum or the work teachers are expected to do. Oh well, economists do talk.

But it’s the system, and systems have inertia with much lag and lead, and here the problem reveals itself. Australia is about 20 years down an education trial of outcome focused education, and this trial is miserably failing.

Forward thinking educators have seen this coming of course, but over ten years ago, on November 22 in 2005 in the Australian newspaper, Geoff Masters made an emphatic defence of outcome focused education. In that article Masters said that it was no longer sufficient to know that teachers are teaching the syllabus. Instead, he advocated for an outcomes focus with measures of what students are learning. At that time, Masters claimed that there was no evidence of a plunge in Australia’s education achievement, as teachers were claiming at the time.

Masters made two missteps.  The first was that he called Australia’s trend too early as it takes 9 years between cycles to establish reliable trends on PISA. This is because, for example, Reading was a major domain in 2000 and again in 2009. Masters made his call after only 5 years of PISA when trends were not yet reliable. The second consequential misstep, is that Masters’ call for a more outcome focused education was heeded, and what started in Victoria during the 1990s as the LAP, and then the AIM, was taken up nationally as NAPLAN in 2008. And here we are, 20 years of outcome focused education, and this mess.

Masters, an eminent exponent of the Rasch model, is of course not to blame for Australia’s demise.  He is informed by an elaborate coterie of interests. This coterie is constructed through cross board memberships and project collaborations, and ranges across interests from the private to the public sector. It includes governments, educational providers and universities.  This coterie has had much to gain from an outcome focused approach to education.  Outcome focused education is ostensibly self-managing, and the move away from the focus on inputs to outputs has seen a demise in the focus of curriculum and ideas. Much of Australia’s meaning making infrastructure has been gutted, and continues to be under attack, from those advocating and strategizing for a continued focus on outcome focused education. These include the demise of subject associations and restricted funding to curriculum bodies.

So as the debate continues, and people start laying the blame, perhaps we should start to focus on the various system coteries and beneficiaries, and not to simply lay the blame at teachers and teacher educators, which are already among the tightest regulated fields in Australia.

Just don’t expect these arguments to come from think tanks and related organisations. Check their boards, their backers and their networks. Unfortunately, don’t expect teachers to articulate it either, for one they are too busy helping students getting band aids, taking music, counselling children, dealing with crises or writing reports. Sure teachers get it, but they get it intuitively, they don’t have the language or the time. They rely and expect system mangers to articulate and argue for them, and on this they have been poorly let down by coteries of vested interest.

The writing genre of responding to international achievement reports in education

I’d like to start by saying that I’m a nice person, not like all those other nasty commentators, by proposing something small and manageable. I just want to highlight that I’m not part of the feeding frenzy created by this latest report.

I’d like to begin by establishing my credentials as someone who cares. I will do this by talking about disadvantaged students identified by this study.  I will do this using devices such as the socioeconomic gradient, and how certain groups are overrepresented in some band or other.  By doing this, you will know I care, and that I’m looking out for the little person, whoever they may be.  I will also be using statistics which establishes my smartness and that I know about these things.

At this point I’m generally out of ideas. But everybody knows that education has teachers as well people teaching those teachers, so I’ll just have a crack at all of them.

It’s good to pick on teachers. School buildings, curriculum, school location and the like are pretty boring. So I’ll avoid them.  I also have no idea how to implement anything, teachers do this right? I don’t understand systems, system managers, public policy and industry structure – that’s for boffins. While I talk at the system-level, I don’t really know what system-level means. Lucky I care so I don’t have to talk like a manager.

Then I’ll drag up some good idea from the past that has faded. I will present them as my own new idea. Or I’ll promote something that will benefit my company or my friend’s company. Maybe something about phonics, assessment, methods of instruction, or just something plain old zany like ‘teach like bushranger’.

I will blame the inertia in the system on all those bad people. You might know them as those other people, those ‘post’ people responsible for post-truth. Whenever I can’t explain myself I will blame teachers and other bad people.

I will finish with a platitude, perhaps a need to look more at the data. Teachers should manage the change though; they get 3 days of professional development per year right? That should do it! Those bad teachers (not the teachers that are reading this, they are good teachers, those other teachers are bad).

My older and younger brother

An allegory on German Philosophy

This story is about my chaotic family, in particular my two older brothers. We will call them the older and younger for now.  My older brother is a very tidy person who values logic and reason but he can be excessively orderly and fussy.  My younger brother is quite a contrast, he’s very likeable and creative but he likes a smoke and a drink and his life can get pretty chaotic.

I could tell that things were going to be different for our family from an early age.  The first time was when the older brother was still a kid.  He went to his best mate Peter’s birthday party and took the younger brother along also. The younger created a real fuss by driving Peter’s mum, along with other women at the party, so mad with his special herbs that she was tricked into cutting off Peter’s head. A pretty violent death for old Pete, and we soon had to slink off home to regroup.

To keep us occupied, our mum gave us a machine to make words and books.  Dr Immanuel came along and gave us some lessons.  The older took a real interest in the book machine. He enjoyed writing about his ideal world and sharing those ideals with others.  But it wasn’t long before the younger started kicking up trying to gain attention by romantically going around dancing, playing music and making art.  Tension remained within the family so mum got Dr Friedrich in.  Dr Friedrich thought our family the ants pants and capable of doing all sorts of things.

I’m not sure how helpful Dr Friedrich was though.  Mum had just bought a new set of kitchen appliances when our neighbour Alf started having trouble with his housemate. My brothers thought it a good idea to just throw the troublesome housemate straight into mum’s new oven.  Dead as doornail, and our neighbour Sam across the road was furious.

Sam gave us a deserved hiding, and while we took our medicine Sam developed his own issues. First Sam became a good friend of the older who helped him make a nice picket fence and some fancy rockets. This soon got boring and then Sam took an interest in the younger.  This led to some good parties for a while with good music and lots of peace and harmony as well as lots of those funny smokes.  But all that came to a sudden end when a bunch of Sam’s drug induced girls started randomly killing people again, in particular an attractive young actress who was pregnant with her first child at the time. It was a horrible end to the party.

Now mum has got us a new toy to occupy us called the internet, the book machine has become boring.  At first this internet was a lot of fun but now tensions are emerging again. The older brother likes the internet for things like study, but he’s also a bit of a snoop and he’s using it to control other people’s lives too.  The younger, well, he’s always liked sex and spends most of his time making and looking at pornography when he’s not whipping up social storms.  It’s all seems like fun for the moment, but I think there will be trouble ahead if mum doesn’t do something.

But I don’t know what mum should do.

My older brother is Apollo, the younger is Dionysus.

 

Eternal Education Headlines

These things we consider known and self-evident and no longer require research effort:

  1. Australian educational performance is precipitously declining
  2. The achievement gap between students from rich and poor backgrounds is growing
  3. Australian students are being left behind by Asian students
  4. Markets are ruining education
  5. Think tanks are ruining education
  6. The Traditional versus Progressive debate will be over when everybody starts listening to me
  7. All educational debates will be over when everybody starts listening to me
  8. Pearson are bad, how can I get a job with them.
  9. Teaching is not that hard, why are people making it so difficult.
  10. If only everybody knew where students are in their learning.

The Uncanny Progressive versus Traditional Debate

Meditation on the following blogs

Dr Beardface  On reading (part 1)  On defending shit work  On ideology

Linda J. Graham  On tax-payer funded research   Angry white men

Greg Ashman   Come, join the enlightenment    Loose ends    The disconnect

debsnet   Traditional Progressivity or Progressive Traditionalism: Ditch the dichotomy

Corinne Campbell  On TeachMeets, EduChats and Marketing

My recent twitter feed has had much discussion about traditional and enlightenment values. While some seem satisfied with their respective positions, to me it’s a manifestation of an underlying discontent that’s been brewing throughout my 30 career in education, and these blogs provide an opportunity to consider these issues propelled by real people with real emotions, not abstract ones.

While the debate had material for many tangential excursions, I will restrict this blog to a couple of key themes – post structuralism, sex and race, the Enlightenment, and the role of teachers in a post traditional landscape. Further, I haven’t addressed all the blogs related to this debate.

This blog is part of my public thinking for my PhD, and the references are as much for my research purposes as for any academic pretensions.  This topic is really too big for twitter and the blogosphere so this is more of an essay, in some ways proving the point that the topic is too dense and complex.

Post Structuralism

Issues with post-structuralism in education drove much of the twitter debate to which I’m responding, perhaps it’s best to quickly summarise my understanding of this endeavour. Structuralism was an attempt to identify underlying structures, codes and conventions that produce meaning and make meaning possible.  However early structuralists like Barthes, Lacan and Foucault recognized that meaning making is not independent of the person making the meaning; that is, a subject’s sex, social class and ethnic identity affect meaning making.  This led to post structuralism and in particular Deconstruction led by Derrida who critiqued hierarchical oppositions in Western thought. Derrida showed that notions such as inside/outside, mind/body, nature/culture were not natural but a construction.  While the work of Deconstruction sought to dismantle and reinscribe textual meaning, it did not seek to destroy meaning. However, in effect, Deconstruction did become a teasing out of warring forces of signification within a text and is therefore associated with broader movements such feminist theory, various psychoanalytic theories, Marxist thought, Post-Colonial Theory, and Minority discourses (Culler, 1997, pp. 125–131)

Posts-structuralism has generated much academic activity and material, and even if a small percentage of this material is dross, this seems to be sufficient to attract much ridicule from traditionalists. Nevertheless, post-structuralism remains a valid and useful endeavour, particularly for education which has a key interest in matters of content and representation. Drawing on the notion of education’s instructional core(Elmore, 1996), students, teachers and content are the three central concerns of education; from a post-structural perspective this translates into two meaning-making subjects and a collection of externally produced content signifiers.  It is the concern with signification and the subject that makes post-structuralism particularly relevant to education, more so than some of the other ‘posts’ related to economics, management, art and culture (e.g. Drucker, 1993; Jameson, 1991). Furthermore, all these ‘posts’ sits within broader changes within western societies sometimes described as a condition of postmodernity (e.g. Harvey, 1990; Lyotard, 1984).

Post-structuralism has failed in many respects to live up to its political promise, while it provides a range of social enquiries it seemed to have had little interest in concrete political issues such as justice, freedom, truth and autonomy (Eagleton, 2008, p. 199). One example is post-structuralism’s scepticism of Government  (see Governmentality e.g. Foucault, Burchell, Gordon, & Miller, 1991; Foucault, 2008). Without recourse to an effective government it becomes difficult to mount a case for emancipatory interests such as equality. A case for equality requires a government capable of both monitoring equality and implementing effective policy in response. Equality requires ‘governmentality’, and universal education is traditionally provided by government. So post-structuralism’s scepticism and critique of the role of government has, unwittingly or otherwise, weakened the position of the state to define, monitor and redress disadvantage in education. Furthermore, in diminishing the state’s role in defining and redressing disadvantage, post-structuralism has, again perhaps unwittingly, opened up the landscape for market forces to redefine and address perceived disadvantage.  Post-structuralism can redress this by either better scoping out its concerns to focus on signification, or by developing a stronger narrative in favour of systems and government.

Irrespective, post-structuralism will continue to have a strong role to play in education due to its concern for signification and representation, particularly when it’s able to take a ‘structuralist’ stance to inform how subject matter should be represented in the digital age. This will continue to be a highly contested area (Beavis, 2010; Kress, 2003). For this reason, post-structuralism is unlikely to be usurped by the more contemporary post-humanism  (Barad, 2003) within the field of education any time soon.

Sex, Race and Uncanny Australia

The trad-prog debate also involved sex and race through the invocation of Angry-White-Men, a reformulation of the post-modern Dead-White-Men. This invocation generated some offence as well as ironic amusement.  There is no doubting the phenomenon of the violent angry male, but there are also men who are angry about other things such as Australia’s treatment of asylum seekers and quality of universal education.  Conflating these forms of anger may not be useful.

Collins Street

Collins St, 5p.m.1955, John Brack © National Gallery of Victoria 

The distinct strata that once divided men and women has evaporated  

The distinct strata that once divided men and women in Australia have also evaporated.  Two of the protagonists propelling the twitter exchange, for example, included a senior female academic and a male student, a reversal of traditional power relations.  These inversions are no longer isolated, Australia’s richest person is now a woman, and we have had a woman prime minister. Further, for each Alan Jones and Andrew Bolt in the public sphere there are is now a corresponding – and arguably more articulate and successful – Clementine Ford and Jane Caro.  Nevertheless, while ‘social media’ power between men and women may have equalised in the public sphere, this has not necessarily translated into real economic or political equality.

Similar issues exist for race. White Australia once was able offer generosity to its Asian neighbours for failed colonisation practices in Vietnam for example. In education this led to a culture of inclusiveness. White Australia is no longer able to assert itself through either colonisation or generosity in the same way from a position of power. Chinese nationals now have the economic upper hand to purchase property within the catchment areas of some of Australia’s most sought after public schools (Chinese buyers flock to Glen Waverley). The Australian economy is no longer able to assert itself within an Occidental context of dominant white Anglo-Saxon men and women. Australia is becoming increasingly dependent on Oriental forces (see Said, 1994 for post-colonial framing of Oriental and Occidental).

So the traditional framings of feminism and post-colonialism are no longer able to provide a coherent narrative of power relations in Australia in a way that resonates with the lived experience of many Australians. Most Australians, particularly in education, now routinely report to by both men and women of both Occidental and Oriental backgrounds.  This is not to say that there are no systemic structural inequities based on sex or race, and that sex and race are no longer valid targets of public policy, but structural inequities can no longer be fully explained in terms of hegemonic white male power.   Nor can white male anger be dismissed as a contemporary manifestation of dead white men, it is likely to be more pernicious than that and involve female protagonists (e.g. Pauline Hanson)

There is therefore unfamiliarity and strangeness around the roles of sex and race in power relations, boundaries that once distinguished one from the other may no longer be tenable or recognisable.  Gelder and Jacobs, drawing on Freud and Kristeva, developed the notion of an uncanny Australia with respect to the sacredness in Aboriginal culture(Gelder & Jacobs, 1998, p. 26).  This notion of uncanniness could be extended to sex and race, an uncanniness that could itself be the root of anger.

A flight to Enlightenment

A flight to Enlightenment and towards the certainty of empiricism is one response to an uncanny Australia and a more complex environment.  While such a flight could be dismissed as a simple psychological defence, it also seems part of a broader trend and therefore worthy of exploration.  For example, Geoff Masters, CEO of Australia’s preeminent educational research organisation, considers the field of educational assessment as currently divided and in disarray due to fault lines occurring between competing philosophies, methods and approaches (Masters, 2013, p. 1). As I have argued elsewhere, Masters’ response to this disarray is a unifying principle that takes a Kantian metaphysical philosophical stance, or an early Enlightenment stance. A stance that presupposes a cognition (presumably white male) before another cognition that acts as a philosophical arbiter of practical reason, judgement, and theoretical reason (Habermas, 1996, p. 2).  Masters’ proposed principle also privileges the role of objective measurement and the Rasch Model (Masters, 1982) of which Masters is a world leading exponent. In doing so Masters also regresses to an early version of the Enlightenment that ignores Hegel’s work in showing that philosophy is not transcendental but historically located (Singer, 2001, p. 13)

Furthermore, education at heart is not a science but a social activity. Education does deal with facts, but mainly deals with norms that are socially constructed.  Facts and Norms should not be confused. Curriculum, for example, cannot be determined by empirical means. Instead, curriculum is developed by drawing on social norms and social reasoning and articulates the shared expectations of a broader community.  Even for those aspects of education that can be measured, the notion of causation is less well understood,  and the validity of meaning making is underdeveloped and under-theorized (Markus & Borsboom, 2013, p. 15). Blind experiments that are able to test some of the more contentious issues are also not possible in education due to ethical constraints, so for many of these issues effective social reasoning is required because empiricism is simply not an option.

While a retreat to the Enlightenment may be comforting and provide certainty in a time of uncanniness, even those dedicated to retrieving the Enlightenment, such as Habermas, emphasise the centrality of moral discourse and pragmatics (Habermas, 1985, 1987, 1996, 1998).

Teachers and Systems

From an effective teachers point of view the dichotomy between traditional and progressive, or any hierarchical oppositions, make little sense. Teachers are practical and pragmatic reasoners who, when given sufficient autonomy and support, use their educational expertise, their engagement with the broader educational community, and their knowledge of their students, to deliver lessons that effortlessly traverse oppositions. It is this skill and experience that makes teachers excellent social reasoners and moral agents. However articulating these skills with systems remains problematic.

Systems provide the resources and administrative authority for teachers to conduct their work, and the system-teacher relationship requires reciprocity.  Where this reciprocity is distorted it can lead to systems colonizing the world of teachers (see Habermas, 1987). One example of where the nature of reciprocity has changed relates to educational standards. Traditionally teachers, as moral agents, contributed significantly to standard setting exercises that reflected social expectations (Cizek, 2012). However, the social process of standard setting is increasingly being replaced by instrumentally defined cut-off points and levels (e.g. see OECD, 2012, pp. 258–263) which may be appropriate for system evaluation but perhaps less so for reporting to students and parents.  The diminishing role of subject associations is evidence of this transition which has led to a weakened relationship between teachers and systems.  System consultations with the teaching profession are being increasingly replaced by private discussions among board-level coteries. There is also the phenomenon of teachers in leadership positions being appropriated (bought, seduced, corrupted) by commercial interests.

So while there is reason to be sanguine about the capacity of teachers to navigate divides within their classrooms, systemic problems remain that require political action to generalise teacher experience across systems.

Looking Ahead

Julia Gillard prime ministership may provide a useful glimpse of what the future might look like. Germaine Greer describes it thus

it’s important to realise that Julia Gillard is part of a coalition. What that means is that she has to negotiate every single policy position. What that means is camel trading on the floor. It happens to be what she’s good at. You can say, ‘We want to know what she really, really believes.’ In fact, it’s irrelevant because whatever she really, really believes is not what’s going to happen.(“Q&A :Politics and porn in a post-feminist world,” 2012)

To me, this is the future. It doesn’t matter what any of us think, it is our capacity to engage and negotiate issues into action that makes us effective.  Who knows what the world would look like when men and women are equal, where the Occident and the Orient are equal, and where the Palestinian-Israeli conflict is resolved.  Nobody knows, and we will only find out if we take the necessary steps forward to engage and negotiate. There is a strong argument to be made that Gillard has been Australia’s most effective Prime Minister, not by way of being able to unify a majority around a single set of ideas in the manner of Bob Hawke, but by way of being able to effectively negotiate difference in a manner that delivered a more enlightened post-traditional society.  Further, Gillard’s post-traditional effectiveness was matched by a traditional hostility including that of Greer, who quickly followed up the above quote with comment that Gillard had a ‘big arse’; probably one of the most disappointing moments in Australia’s gender debate. Almost uncanny.

Barad, K. (2003). Posthumanist performativity : Toward an understanding of how matter comes to matter. Signs, 28(3), 801–831.

Beavis, C. A. (2010). English in the Digital Age: Making English Digital. English in Australia, 45(2), 21–30. Retrieved from http://www98.griffith.edu.au/dspace/handle/10072/37149

Cizek, G. J. (Ed.). (2012). Setting Performance Standards : Foundations, Methods, and Innovations. New York: Routledge.

Culler, J. (1997). Literary Theory: A Very Short Introduction. Oxford: Oxford University Press, UK.

Drucker, P. (1993). Post-Capitalist Society. Routledge.

Eagleton, T. (2008). Literary Theory : An Introduction, Anniversary Edition. Minneapolis: University of Minnesota Press.

Elmore, R. F. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1), 1–26.

Foucault, M. (2008). The Birth of Biopolitics: Lectures at the Collège de France, 1978–1979. (G. Burchell, Trans., A. I. Davidson, Ed.). New York: Palgrave Macmillan.

Foucault, M., Burchell, G., Gordon, C., & Miller, P. (1991). The Foucault Effect: Studies in Governmentality. Chicago: University of Chicago Press.

Gelder, K., & Jacobs, J. M. (1998). Uncanny Australia: Sacredness and Identity in a Postcolonial Nation. Carlton South: Melbourne University Press.

Habermas, J. (1985). The Theory of Communicative Action: Reason and the rationalization of society. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1987). Lifeworld and system: a critique of functionalist reason. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1996). Moral Consciousness and Communicative Action. (C. Lenhardt & S. W. Nicholsen, Trans.). Cambridge MA: MIT Press.

Habermas, J. (1998). Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. (W. Rehg, Trans.). Cambridge MA: MIT Press.

Harvey, D. (1990). The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change. Cambridge MA: Blackwell.

Jameson, F. (1991). Postmodernism, Or, The Cultural Logic of Late Capitalism. Duke University Press.

Kress, G. (2003). Literacy in the New Media Age. London: Routledge.

Lyotard, J.-F. (1984). The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press.

Markus, K. A., & Borsboom, D. (2013). Frontiers of Test Validity Theory : Measurement, Causation, and Meaning. New York: Routledge.

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. doi:10.1007/BF02296272

Masters, G. N. (2013). Reforming Educational Assessment: Imperatives, principles and challenges. Australian Education Review. Retrieved from http://research.acer.edu.au/aer/12

OECD. (2012). PISA 2009 Technical Report. Paris: OECD Publishing. doi:10.1787/9789264167872-en

Q&A :Politics and porn in a post-feminist world. (2012). Australia: Australian Broadcasting Corporation. Retrieved from http://www.abc.net.au/tv/qanda/txt/s3451584.htm

Said, E. W. (1994). Orientalism. New York: Vintage Books.

Singer, P. (2001). Hegel: A Very Short Introduction. Oxford: OUP Oxford.