Outcome focused education is not so much an approach or philosophy. It is remarkable for what it lacks and is akin to Adam Smith’s ‘invisible hand’. It has two components, neoclassical economics and Rasch-based measures. It lacks a focus on curriculum.
The knowledge and skills valued by society are traditionally determined by policy-makers in consultation with stakeholders who implement systems to develop and deliver curricula consistent with community needs, expectations and values. Teachers, relevant organisations, and government authorities work together to generate shared meanings that are expressed through curriculum. However, it is these activities that are increasingly missing from an outcome focused approach to education. This void becomes manifest when educational issues are identified and solutions are proffered from overseas.
Seminal developments in both the Rasch Model and neoclassical economics come out of Chicago. While George Rasch was originally from Denmark, it was at the University of Chicago that researchers such as George Wright and Australia’s Geoff Masters made seminal developments during the 1980s. The Rasch model is mathematically elegant, and perhaps even beautiful in its simplicity and generalisability. It is a model that informs assessment measures using link items with wide horizons. PISA measures, for example, have horizons over participating countries and over time. Similarly, NAPLAN measures have horizons over Australian states and over time. For trend reporting, these horizons are necessarily anchored, and for PISA and NAPLAN they are anchored on conceptions of knowledge defined in the initial cycles of PISA. So while the measures were good at the time, and remain very relevant, they haven’t evolved much in consideration with things such as the internet and graphic calculators.
While PISA initially made a big effort with its testing material, good content is not really required for using the Rasch Model. George Wright lauded the Rasch model on the basis that it transcends the questions, and transcends the measuring instrument. So educational content is not so important for Rasch. This is reflected in NAPLAN for example, with ACARA only claiming that NAPLAN broadly reflects aspects of literacy and numeracy within the curriculum across the states and territories. So NAPLAN doesn’t actually assess curriculum, just somewhat assesses curriculum. Due to the need to report trends, NAPLAN is anchored in aspects of curriculum from several years ago.
The beauty and the pernicious of the Rasch model is exemplified in the diagram below. It illustrates two students doing a rudimentary mathematics test. Student A can do all four operations but only with small numbers, Student B can only do two operations but with both high and low numbers. Yet their scores are the same, and their Rasch measures would also be the same.
What is remarkable about this example is that based on the Rasch scores these students are the same. That is, in more complex assessments such as NAPLAN with a set of broader tasks, similarly varying skill profiles are not able to be detected. Rasch measures are not useful for determining what a teacher should teach next. These measures are only useful at a system level. Teachers are left to scramble in the dark. They may be implored to “dig into the data”, but that well is really dry; there is no more information there. NAPLAN doesn’t provide the sort of information a teacher needs to target or differentiate teaching.
NAPLAN and PISA are useful for system management in a broad brush economic sense, they provide a means for bureaucrats, think tanks and pundits to meaningfully talk about education. They provide a link to neoclassical economics by providing proxies for output in the supply and demand equation. They are used in reports by Gonski and the Productivity Commission. They are also used by think tanks such as the Grattan Institute, and Mitchell Institute. It is noticeable that these reports generally lack reference to curriculum documents or pedagogy. In this way, think tank commentary is both out of date and out of context. It is meaningful in that these data are the best data available at the system level, but most commentary is vacuous discussion about numbers, not meaning. Everybody feels like an economist when they talk using PISA and NAPLAN data, and nobody feels like a teacher.
NAPLAN and PISA are not useful for talking about students or teachers, mainly because these tests don’t reflect the work that they are engaged in. Teachers and students are engaged in curriculum, while neither NAPLAN or PISA are.
Teaching is a rare profession from a Human Resource perspective. Their work is judged on measures that have nothing to do with their roles. Further, any overt attempts at achieve targets is frowned upon, even called cheating. I often wonder how long it will be before this house of cards is formally challenged in the courts.