More confected ATAR outrage – is it time for a grey haired sage?

The ongoing furore surrounding the ATAR is based on confected outrage rather than reason.  While the outrage is based on small grains of truth, the outrage also reveals misunderstandings and contradictions that are particularly destructive.

There are calls for an ATAR crackdown, and apparently Minister Birmingham is set to announce one soon. But what does this mean? Is it more bureaucracy, more centralisation, or just time to bring in a grey haired sage to contain and placate the debate?

ATAR is a rank, not a standard

Each ATAR is derived from a statistical process and gives a ‘best guess’ of where each student is in relation to all other students in the cohort.  The ATAR is a rank that makes no reference to any standard.  Reference to a standard is not possible as the ATAR represent studies undertaken in different state jurisdictions each with different subject descriptions and assessment regimes.

So an ATAR score of 30 is only an indication, or best guess, of where a student is in relation to the student body, and not what a student has achieved or is able to do. So when Asafa Powell came last in the 2012 Olympics 100 metre final, his rank was in the bottom 20 percent for that race, but that still makes him a pretty fast runner.

Further, as an annual ranking mechanism, the ATAR is not able to monitor any trends of student achievement against objective standards. Such objective evaluations can only be conducted by programs such as TIMSS, PISA and NAPLAN.

A national exit credential that explicitly reports against a standard is possible, but this would require a considerable bureaucracy and agreement among the states.

Politicians, bureaucrats and commentators who conflate ATAR with explicit educational standards are simply wrong. The ATAR only provides a fairly derived indicator of rank.

Politicians, bureaucrats and commentators who seek to implement national standards at the secondary school level are in fact calling for the establishment of a large new bureaucratic process. Such proposals have merit but these need to be balanced against other priorities.

ATAR is an indicator that does not tell the whole story

There is a persistent contradiction in much of the commentary of the ATAR.

On the one hand there is an argument that the ATAR is an indicator that does not provide an accurate indication of student potential. This view is supported by the comments of various vice-chancellors (Dawkins VUKristjanson Swinburne) who argue that evidence in the form of interviews and portfolios are also considered for selection.  It is the fact that some universities are willing to consider this additional information which can make published ATAR cut off score an unreliable indicator of the effort required by students to secure selection.

That the ATAR is simply an indicator that does not tell the whole story makes the critique of universities who admit students with low ATARs meaningless.  That is, when a student with a low ATAR is admitted to a university course the likelihood is that they have been able to gain admission through the provision of additional evidence.

Politicians, bureaucrats and commentators who argue that the ATAR is a poor indicator of student achievement and at the same time criticise universities for admitting students with a low ATAR are being inconsistent.

Autonomy versus centralisation

The ATAR accommodates credentials across all the states and territories,each with their own subject descriptions and assessment regimes.  The ATAR is provided to students from public, independent and Catholic schools. The ATAR also incorporates the International Baccalaureate and range of other candidates who may not have  participated in a Year 12 credential. The ATAR is used by the full range of universities across Australia.

The ATAR is therefore a meeting place for a range of autonomous individuals, institutions and systems.   Students are free to choose the universities and courses to which they wish to apply, schools are free to prepare their students in any way they see fit, universities are free to select the candidates in the way they see fit. The ATAR therefore provides a transparent means of connecting autonomous individuals.

Politicians, bureaucrats and commentators who demand that universities enforce arbitrary admission standards, or the call for the establishment of common standards, are in fact calling for increased centralisation of control and for the establishment of large national bureaucracies. This runs against calls for more autonomy and needs to be called out as such.

The current ATAR provides an efficient way of linking prospective students with universities. It is a mechanism that respects the autonomy of the states, schools, students and universities.  Demands to change the process are only likely to lead to increased national bureaucracies.

ATAR is for students and no one else

The ATAR provides enfranchisement for students into tertiary admission.  It provides an efficient way of linking students – each with their own emerging hopes, dreams and aspirations – with universities – each with their own mission and vision.  The power of this enfranchisement is demonstrated by the celebrated efforts of Casimira Tipiloura, the first Tiwi Islands student to graduate with and ATAR (see SMH article).  This enfranchisement should not be appropriated by schools for marketing, by universities for marketing, or by politicians and pundits for cheap political gain.

Of course the ATAR can be improved

Of course the ATAR can be improved in terms of the information that it provides to universities and the information it provides back to prospective students. Its fairness and generalizability could also be improved. To achieve these aims would be consistent with Australia’s aspirations to become a knowledge economy. It would also require additional bureaucracies, centralization and agreement across the the states. However the current destructive criticism of the ATAR is only consistent with Australia’s sliding standings within the international educational community.

So the ATAR is irrelevant –what next?

Reflections on ATARs are irrelevant, vice-chancellors say – by Henrietta Cook

The demolition of the ATAR seems well under way, so it may be prudent to look ahead to see what may happen, and even what may already be happening.

At first glance the demise of the ATAR will do away with many of the unpleasant aspects of the transition from secondary to tertiary education.  With the ATAR gone so too will the fierce competition across large cohorts, the unreliable published cut-off scores, the need to undertake a broad curriculum, and the ignominy of being reduced to a single number.  But what else will we lose?

With the ATAR under attack a number of foundational aspects of our education system are also under attack: a broad curriculum, portability of aspirations, equity and fairness, and transparency.

A broad curriculum, such as Victoria’s VCE, initiates students into broader cultural aspects of society.  The link that ATAR provides between the VCE and university motivates students to undertake a broad curriculum.  Alternative university selection regimes, such as those used for medicine, tend to be uni-dimensional focussing on limited curricula.  An increase in the use of these uni-dimensional assessments for tertiary selection will lead to a contraction of secondary school curricula; noting that the VCE currently has over 100 subjects including community languages.

Student aspirations can change from the time they commence VCE.  By undertaking subjects that a student is good at and likes within a generic ATAR framework provides students with flexibility.  A student may finish Year 10 wanting to become a doctor and undertake a VCE with a science focus. If that student changes their mind during the VCE to become a lawyer, for example, that can be readily accommodated within the ATAR.  The demise of a common generic ATAR will only give rise to costly proprietary selection tests for sought after university courses.  Each proprietary selection test will require their own preparation regimes which will make it harder for students to change aspirations during their VCE, their gap year, or even early tertiary studies.

Equity and fairness are paramount in the current VCE and ATAR process.  Within the VCE students are able to demonstrate their effort and competence in a range of subjects and through a wide range of assessment types.  The VCE processes – including double marking and moderation between school based and central assessments – provide for fair scores and ATARs that are blind to matters such as socio-economic and cultural background.  While processes such as interviews and portfolios are likely to provide relevant information for university selection, they are also more prone to selection bias.  For example, it’s not hard to imagine a dilemma for a medical degree selection panel when two students are equal except background. If one candidate’s parents were both doctors while the other unskilled refugees we might all imagine which one the panel would choose. But the transparency of that selection will diminish, with a potential loss of equity.

Given that the ATAR is common to most secondary school students in Australia it provides a great source of population statistics for monitoring things such as equity. It also provides some guidance to students on where different university courses sit in terms of demand.  Although these guides are necessarily imprecise given that universities supplement ATARs with interviews and the like.  The ATAR also allows for the tracking of students to enable reporting on matters such as equity. That is, while the awarding of the ATAR maybe fair in terms educational achievement, the distribution of the ATAR may reveal certain inequities in our education system. The ATAR thereby provides a valuable input into political discourse.  The abandonment of the ATAR, and a move towards proprietary selection regimes, will remove this transparency as the requirement to report will diminish.  As the ATAR diminishes Australians will have diminished political recourse to tertiary selection and instead will need to seek market remedies to selection issues. This does not auger well for a fair society.

The roots of the ATAR go back to the 1960s, a time of full employment and high demand for university graduates.  Merit, equity and transparency were becoming important as Australia saw itself as becoming more egalitarian leaving behind its class-based English roots. It was a time when any student with ability and initiative could become whatever they wanted to become regardless of background.  This led to a prosperous and socially mobile generation. The 2010s are different. Employment is a lot softer and the clear career options of fifty years ago no longer exist. The focus on merit, equity and transparency has moved to more market driven dynamics.  This has led to an increased focus on gaming the system and attaining branded cache rather the pursuit of knowledge and personal aspirations.

The power of the ATAR to enfranchise students and to link them to a vast world of possibilities is demonstrated by the efforts of Casimira Tipiloura (see SMH article). Casimira is the first Tiwi Islands student to graduate with and ATAR.  The sense of celebration accompanying this effort surely points to what Australia is, and could become.

Australia’s education system is being put in a precarious place with the sustained attack on the ATAR. The easy option is to ditch the ATAR and to leave it to an open market. This is likely to lead to diminished transparency and fairness. Let’s hope that instead effort will be taken to ensure that the ATAR and its functions are improved so that Australia remains the country of the fair go. Something that we can no longer take for granted.

ATAR – it’s about fairness, not prediction

There is a tradition across the political spectrum to attack institutions like the ATAR.  These attacks come in two flavours. The ‘right’ attack elites for imposing artificial social engineering, the ‘left’ attack systems for imposing arbitrary rules that erode liberty. Then there are commentators who confound these arguments to generate media storms of no substance.

We are currently witnessing a flurry of attacks on the ATAR, an institution that has evolved over several decades to provide exiting secondary students with enfranchisement to Australia’s tertiary educational intuitions.  The ATAR provides a relatively transparent national clearinghouse where students can use their secondary schooling outcomes to open up conversations with tertiary institutions throughout the country.

A consistent complaint about the ATAR relates to its lack of predictive power for future academic success.  This complaint demonstrates a fundamental misconception of educational effort. When teachers and students make an above average effort to achieve an ATAR they are in effect seeking to defeat predicted behaviour.  Similarly for universities where students and lecturers make varying degrees of effort to defeat predicated behaviour. The motivation for much educational effort is social mobility, to transcend one’s background for a more enlightened and prosperous future. It is the variability of this educational effort that makes the ATAR a less than ideal predictor.  What the ATAR seeks to do is fairly recognise educational effort.

The demand that ATAR be an accurate predictor of future academic success is a dangerous one.  From a statistical point of view there are a range of background variables that can predict academic success. These include sex, socio-economic background, ethnicity, and a range of other factors. These variables are reliably used by studies such as PISA and TIMSS to evaluate the fairness of education systems. These variables are also often used in value-added models and measures, notwithstanding that there are many cautions on the use of such measures (see AERA).  These variables could also be used to enhance the predictive validity of tertiary selection.  But enhancing the predictive validity of tertiary selection through such psychometric techniques would only deny opportunities for students with disadvantage who make an effort to overcome that disadvantage.

Another feature of the ATAR is that it looks to both the past and to a future.  Primary and secondary schooling is about more than further education and careers.  A sound education seeks to make well-rounded citizens capable of making a civil contribution as well as being capable of living subjective meaningful lives. So when a student reads Shakespeare as part of a secondary level English credential it is read not solely for furthering educational and career prospects but also to be initiated  into broader aspects of society. By valuing educational effort across a broad range of curriculum the ATAR contributes to a civil society. This contrasts to the growing trend of uni-dimensional selection regimes.

As an institution open to public scrutiny and accountability, the ATAR is highly visible and easy to attack for those seeking a sensational story. The public transparency of the ATAR contrasts to the opaque nature of competing selection regimes, such as those used in fields as medicine, that can be costly and which play no role in informing the broader national debate on educational equity.  Proprietary selection regimes also tend to be uni-dimensional with a greater focus on predictive validity rather than broader learning.  As the use of these proprietary selection regimes increases the transparency and public accountability will diminish as will the broad curriculum of secondary education in Australia.

Many attacks on public institutions can be framed in the psychological terms of the present and munificent ’good mother’ and the absent and scarce’bad mother’ that we experience as children  As we mature, we integrate these two views and recognize the competing priorities  into an integrated whole. Many of the attacks on the ATAR simply seek to kill the bad, but in doing so will also kill of the good emancipatory interests that the ATAR provides for.  The death of the ATAR would simply leave students to the vagaries of wolves in the marketplace, or throwing out the baby with the bathwater.

The competing interests of diversity and uniformity underpin much of the misunderstanding of the ATAR.  The ATAR seeks to integrate the idiosyncratic nature of each state’s secondary credentialing regime.   There are different subjects, different assessment processes, and different reporting mechanisms within each state.  There are independent schools, public schools and Catholic schools that need to be catered for.  There are also non-state based credentialing institutions such as the International Baccalaureate that need to be integrated into the ATAR.  Then there are mature age students, and others who haven’t completed a Year 12, who also seek an ATAR.  The ATAR needs to accommodate all this diversity into a single ranking useful for universities.

Another misunderstanding is that the ATAR is a monolithic hegemonic system. Instead, the ATAR is a central clearing house where prospective students and institutions connect.  Within this system students and institutions have autonomy. The ATAR exists to provide a streamlined conversation between prospective students and institutions.  Some institutions, with the motivation and resources, will supplement information provided by the ATAR with reviews of portfolios, through auditions, and interviews. These conversations may also involve discussion of various pathways that a student might undertake to achieve their goals. These negotiated outcomes that consider both the needs of the student and institution make the published ATAR cut-off necessarily unreliable.  Furthermore, the ATAR can make no demand for universities to use it fairly. Universities are held to account through their own structures.

The ATAR is not there to be gamed, students seeking to pursue a course or a profession should do so on the basis of what goal they are seeking to achieve. No matter what ATAR is attained there is likely to be a pathway to pursue that goal for students with the aptitude and disposition.  Those seeking to gain prestige by selecting a course on the basis of a published ATAR cut-off are likely to be disappointed.

The ATAR is a crude instrument, it is not a predictor of student success, it is not a measure of value-add by the school, it is not a reliable method for evaluating university courses.  There are other methods for that. Each ATAR belongs to each student in the first instance, as but one signpost in life’s journey of hopes and dreams.  The ATAR should not be perversely appropriated by others for political, marketing or commercial purposes.

Time for the emeriti to stop taking pot shots at the establishment

A rejoinder to Richard Teese – How our elite unis ‘game’ the VCE and ATAR

Richard Teese’s article on universities, the ATAR and VCE shows that Hebert Marcuse’s Great Refusal is alive and well among Australia’s elite ‘progressive’ thinkers. The Great Refusal continues to provide the vacuum that allows neoliberalism to flourish. Teese’s article, with its Great Refusal tinged with a bit of ‘dead-white-male’ transcendence provides a caricature of authority that leaps established institutions at a single bound, swiping hard working teachers along the way. Teese confounds and conflates many distinct interests, institutions and dynamics in his overarching spray at public institutions; in particular, the role of universities, the ATAR and the VCE.

Teese’s authority on universities must be respected however. When he characterises their behaviour as bellicose, vain and market driven we need to accept that these are true even if somewhat localised and self-reflexive by way of Teese’s own university career.  Yet these characteristics may not be attributable to other universities with their autonomous mission and values.  Saying that universities ‘hide behind their students’, instead of taking ‘pride in their students’ could reflect Teese’s personal university experience, or may perhaps reflect a personal negative disposition.  Nevertheless, each university has its own autonomous body politic consisting of its own academics, so sweeping generalizations about their collective motivations are likely to be erroneous.

As to the ATAR, it is what it is. It provides a common meeting point for individuals and institutions to coordinate their actions.  The power of enfranchisement that the ATAR can bring is demonstrated by the celebration of Casimira Tipiloura’s achievement in being the first from the Tiwi Islands to attain an ATAR.  The ATAR provides students with a personal indicator that allows them to open up conversations with a large range of institutions on future educational options.   Of course how individual universities take up that conversation is up to them.  Some are indeed odious by priding themselves on the basis of QS rankings, research rankings, and world university rankings.  While the ATAR is a good facilitator of conversations between institutions and students, it remains a crude instrument and necessarily so given its academic and geographic scope as well as its focus on inclusiveness and fairness.  Furthermore, it is an annual ranking that’s not criterion referenced so substantive meaning of rank is not stable from year-to-year.  The ATAR therefore has no predictive validity and is not designed to be a predictive measure.  As Teese identifies the progress of students at the tertiary level is dependent on the preparedness of both students and academics as well the university’s balanced priority between teaching and research. Commentary on the ATAR’s predictive validity is therefore ill informed.

The most pernicious aspect of Teese’s commentary relates to the VCE.  The VCE remains a world leader as a broad based credential.  It has over one hundred subjects including community languages, has a wide range of assessment types, has a mix of internal and central assessment, has substantial teacher involvement in implementation at the school and central levels, and uses effective statistical techniques to articulate student VCE achievement with national and international institutions.  The VCE involves detailed processes conducted within tight timeframes on a shoe string budget.  Criticism in the vein of Teese’s can only damage these highly effective yet fragile processes operating within the public sphere. Teese’s criticism can only provide succour to neoliberal forces who would seek to apply proprietary uni-dimensional models to tertiary selection.

There are of course significant broader equity issues in education and these have been well identified by studies such as the OECD’s PISA.  It is interesting to note that the rearticulation and regurgitation of these same issues continues to be of interest to think tanks, research centres and academics alike. However, callous criticism of public and civil institutions such as the VCE and ATAR can only undermine the universal enfranchisement that these institutions seek to provide. It is a type of criticism that serves to dissolve publicly justified processes to have them replaced by opaque market based mechanisms.

The tradition of the Great Refusal is therefore alive and well and in some part explains why we have reached this current situation.  Victoria, a state with a university ranked among the best in the world for education, a university with academics that wantonly slag-off fellow state-based institutions, a state with a declining educational performance. I will not be as careless as Richard Teese with my inferences however.

What is clear is that as societies become more complex knowledge becomes increasingly differentiated, concurrent to this phenomenon are demands for broader enfranchisement for students of all backgrounds.  This requires enhanced technical and problem-solving skills across the educational industry to develop ever more comprehensive systems that are justifiably fair in ensuring enfranchisement at the national clearing house that is currently the ATAR. Unfortunately the current academic focus seems to be on cheap pot shots at the establishment, a stance that should have died in Paris in 1968.