On teaching as a profession

Reflections on Is teaching a profession?

The idea that the term ‘teacher’ be universally understood seems sensible; particularly if the term ‘teacher’ is used in conjunction with qualifiers such as ‘trained’ or ‘accredited’.  While this may seem sensible and consistent with community expectations, the means for achieving a universal understanding is problematic in a pluralist society.

Universality has many appeals.  It allows for teachers, deemed ‘accredited’ in one context, to transfer jobs to a context sharing that accreditation process. It also assists students to develop a consistent understanding of what a teacher is so that they can match those expectations.

That universality is too controlling and prescriptive is one anticipated criticism. Indeed, universality could be seen as a return for direct instruction, explicit teaching, and highly structured curricula and syllabi.  Such prescriptions would indeed de-professionalise teaching as a profession.  So a more open definition of universality needs to be developed, a definition that’s able to capture diversity and difference and leaves the way open for certain forms of innovation. Claims for universality should not simply be on the basis of asserting a particular way of doing things, a particularly methodology, or even a particular ideology.

My working hypothesis/principle regarding universality is that a competent teacher acts in a way that other competent teachers would agree with. This is a working hypothesis only and will need to be adjusted and developed as I work though it.  It follows the work of Habermas and his principle of universalisation and principle of discourse ethics(Habermas, 1996).  My working hypothesis/principle allows for diversity of methods and outlooks within a broadly agreed framework.

My working principle relies on the development of norms for education. As I have argued elsewhere [norms and teachers], in my experience norms and standards provide for prosperity. Further, all successful innovative project I’ve worked on became routine and successful once norms were agreed across participants. However, the creation of norms is not straight forward in a plural society undergoing significant structural transformations at a number of levels.  Education is responding to demands from feminist theory, minority discourses, post-colonial discourses, new means of wealth creation, new means of production, and emerging technologies. All these influences make valid claims of education and need to be incorporated into new norms.

Teacher-led movements are a good way to develop new norms that are consistent with broader societal expectations. Unfortunately the involvement of teacher led institutions in education has diminished in recent decades, particularly the role of subject associations. Again following Habermas (Habermas, 1987), much of the atrophy in teacher participation is due to colonization by the system (non school actors) of the lifeworld of the classroom. Instrumental reasoning (e.g. uni-dimensional assessment) , strategic management, and expert coteries have subsumed many forms of teacher engagement. Grassroots movements such as researchED, Northern Rocks, WomenEd, teachmeets, as well as online forums are therefore a welcome development that needs to be built upon.

Teacher-led institutions are essential for vibrant education systems. Systems and central bureaucracies also need to engage with teacher groups as many new norms will need central and institutional support to implement. These include norms related to curriculum, and norms requiring regulation or legislation such as those pertaining to privacy and technology. Other norms require central support in terms of technological and infrastructure provision.  In return for this support, by way of reciprocity, teachers will need to come to table being able to constructively engage with a range of perspectives in order to work towards solutions that are agreeable to all.

Habermas (Habermas, 1985) also alerts us to the dangers of regressing to instrumental or technical reasoning when reasoning about disagreements. Habermas suggests that in coming to an agreed understanding participants need to engage three worlds; the technical or objective world, the inter-subjective world of norms, and the personal world of integrity and intent.  Learning styles provides a good practical example. For example, much research indicates that learning styles have no efficacy in terms of learning outcomes, nevertheless most people do indicate a preference for a particular style (see Pashler, McDaniel, Rohrer, & Bjork, 2008). So from a technical perspective learning styles may be of no use, where as from a perspective of norms it may be seen as important in terms of generosity. Why serve a cup of tea if someone prefers a coffee? So perspectives are important and make discussion more complex. Furthermore, validity around technical reasoning is becoming increasingly more sophisticated, particularly in relation to the validity of measurement, causation, and meaning (e.g. see Markus & Borsboom, 2013; Newton & Shaw, 2014). These developments increase the cognitive load on norm setting participants.

While teacher involvement through teacher-led associations and central office working parties is welcome, care needs to be taken to ensure that such groups do not descend into group think or other forms of balkanisation.  These groups need to be broadly representative, and representatives need to be competent to participate. Society is becoming increasingly complex and knowledge becoming increasingly differentiated. Education needs to respond to these changes. Teachers need to be in the forefront of these transformations.

Habermas, J. (1985). The Theory of Communicative Action: Reason and the rationalization of society. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1987). Lifeworld and system: a critique of functionalist reason. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1996). Moral Consciousness and Communicative Action. (C. Lenhardt & S. W. Nicholsen, Trans.). Cambridge MA: MIT Press.

Markus, K. A., & Borsboom, D. (2013). Frontiers of Test Validity Theory : Measurement, Causation, and Meaning. New York: Routledge.

Newton, P. E., & Shaw, S. D. (2014). Validity in Educational and Psychological Assessment. London: SAGE.

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning Styles. Psychological Science in the Public Interest, 3, 105–119.

Norms and Teachers

In response to a short twitter conversation, that required a fuller explanation.

When I first started teaching in the 1980s I was assigned to a school with declining enrolments but good infrastructure. In particular there was a large ‘open span’ classroom for over 100 students, the legacy of a group of innovative team-teaching maths teachers from the seventies. When I started these teachers had left, and the school had reverted to a one teacher, one class, one room setup. I have since observed several cycles between open plan including team teaching and regular classrooms, a norm to which it always seems to revert.  This scenario is emblematic; however an increasing urgency is developing around norms in education.

What norms can do – the heyday!

Norms come from practical and social influences and we may not always recognise them. After World War 2, and arising from a practical economic imperative, many Victorian children were taught in grey Light Timber Construction Schools.  In hindsight these were drab, but their standardisation allowed them to be quickly built across the state.  We also have educational content norms such as curriculum and the common books we read. There are also social norms related to what culture we consume such the films we watch and bands we listen to.  These norms provided for a social cohesive and prosperous post war generation.

One norm that I would like to focus on a bit more is the humble A4 paper.  Some like myself still remember foolscap, but the A4 standard won the day. This allowed printers, paper suppliers, photocopy manufacturers and a range of distributors to streamline operations. It also gave us a common understanding of fairness on exams and NAPLAN tests that are in standard white A4 for all students, other than perhaps for special supplements.

The norms we once took for granted are becoming less relevant, and new norms need to be created. Over the decades, educational norms have been challenged by developments such as feminist theory and minority discourses.  Norms are also being challenged in a massive way by technology, from the way that classrooms are physically organised to the way that educational content is delivered.  But required new norms are not being created or manifest as proprietary patents or trade secrets.  There is a wild west in technology towards norms and standards.

When norms fail

Some spectacular examples of failures in norms include technology problems with the GSCE exams in the UK, the Apple-Pearson curriculum in the US, and the Utranet in Australia. All these projects involved achievable technology, but I contend failed due a lack of appropriate norms.

A more spectacular example include the difficulty of getting high stakes exams on computers.  Exams such as the Victorian Certificate of Education (VCE) are massive logistic exercises requiring coordination across all VCE teachers and schools, central offices, printers, and related stakeholders and suppliers.  There are established norms for the size of paper and how to behave in terms of seating and not copying.  These established norms do not apply to computer-based exams, and new ones need to be created. How will fairness be managed with NAPLAN on BYOD where students with better computers may be at an advantage. Time will tell, whenever computer-based NAPLAN is implemented.

But these are small examples of a massive range of new norms that need to be developed for technology-based education. In particular new norms for how we manage space and time in education.

The Role of Teachers

Drawing on the work of R.S Peters, a key purpose of education is initiation into society that involves both cultural transmission and cultural regeneration. Teachers are central to this process of shaping, moulding and growing.  To abrogate this responsibility is to leave students in a technological ‘Lord of the Flies’ limbo, we cannot leave this to students to decide. No matter how complex this task, this moment in history requires teachers to be particularly involved in creating the environment for students to be socialised into broader society so that they can live happy and fulfilled lives.

How to Create Norms

Norms are about broad agreement and common understanding, they cannot be imposed. Wherever teachers withdraw their participation coteries of experts and commercial interests are more than happy to step in. However, these groups are not as well placed to inform the development of norms as teachers. Teachers know how students work, teachers are aware of antecedent norms and their workings, teachers know what’s practical, and teachers tend to better clued in to society including what business wants.  So teachers are well placed.

Teachers therefore need to make greater demands for the establishment of, and participation in, government-based working groups and committees to establish new norms. These norms need to address new usages of technology including general specifications for new learning spaces and technology-based curriculum. This may be expensive, but it is likely to be more effective than clueless expert coteries and solo innovators.

I have been involved in a small number of successful innovative projects from the classroom through to the international level. In hindsight, the success of each of these projects rested on the establishment of agreed norms across participants. Norms are as important as facts.

On Measuring Global Reading Progress

Reflections and comments on ACER’s Next steps: measuring reading progress

Strong governments and strong global institutions are important for defining, monitoring and addressing inequality in education.  These three policy activities are linked by policy narratives that need to be strong, coherent and consistent to garner global legitimacy. The efforts of ACER’s Centre for Global Education and Monitoring is to be commended on this front.

Work on the United Nation’s Sustainable Global Development (SGD) goals goes back to 1972, and its 2030 agenda focusing on Poverty, Food, Health, Education, Gender and Water is laudable and an agenda to which we could all agree.  However agreement on implementation requires the legitimation of a stronger narrative, and there are elements of the ACER approach that I would like to explore in this blog.

There are many things to like about ACER’s approach; the use of Item Response Theory (IRT) to develop a commonly agreed scale is one of them. IRT is a proven methodology for system and national evaluation, even though the methodology becomes suspect at the school, class and student levels.  The other welcome element is the use of pairwise comparisons in the development of content. The recent increase in the use of teacher-based pairwise comparison is welcome because it reengages the teaching profession with scale formation, an engagement that has atrophied over recent decades due to the use of IRT scaling methodologies. However, in the ACER proposal teacher engagement seems limited to pairwise comparisons in preliminary item selection, and does not seem to extend to international agreement on content.

Where the proposal is likely to encounter legitimation issues relate to the hypothesis that educational skills are universal across the target countries and able to be described on a common scale.  Sure, technically this can be done, I’ve rarely seen any test data that doesn’t scale, and where some items don’t scale properly these can be removed for ‘mysterious item reasons’. However there is bound to be concern around the legitimacy of claims about the universality of scales developed in this manner.

As I have argued elsewhere [on a unifying principle], the notion of being able to universally ‘identify where a student is’ is problematic. There are many ways of describing this issue. One way is to say that it’s too Kantian and ignores the work of Hegel in showing that knowledge is historically and socially located, and the work of Marxists that shows that formulations of knowledge can reinforce disadvantage.  Another way is to describe the approach as too metaphysical by presupposing a universal Cartesian space in which students can be located. Realism is yet another word that comes to mind, an approach that assumes that what IRT measures actually exists in reality.  Again, as I have argued elsewhere [constellation and continuum], the continuum metaphor is only one way to describe learning progress. So the observation that ‘progression occurs in a somewhat lumpy way’, is more than likely a reflection of the IRT model or metaphor, and not a phenomenon from the underlying reality of learning.  This is not to discredit the validity of the IRT model or results derived from it for the purpose of international evaluation; it simply questions the universality of any claims made.

An alternative to presupposing universal realism across nations and cultures on matters such as reading and mathematics, is to develop a procedure for SGD countries to agree on what is common to all with respect to these content areas and to create a common scale around that agreed content and then report explicitly to that effect. That is, report that the scales represent what has agreed to be common, and not was is considered universal and enduring.  The claims to universality, along with the described content methodology, could be characterised as cultural appropriation followed by cultural imperialism. Such an approach is likely to meet with resistance from teachers and the like at some point. People are social and cultural beings who use language to express themselves socially and culturally.  Of course reading progress is important to these expressions and for prosperity, but these expressions are also specific to each cultural context and not a universal function of language.

French President Charles de Gaulle’s famous 1962 observation on “How can you govern a country which has two hundred and forty-six varieties of cheese?” provides a good example of where language equivalence does not mean cultural equivalence. Cheese (Australian), kaas (Dutch) and fromage (French) are language equivalents, but Australians have Cheddar and Tasty, the Dutch have Edam and Gouda, and the French have a much broader variety.  Claims to social and cultural equivalence based on the simple language equivalence of ‘cheese’ is therefore likely to meet resistance. Reporting with claims to universality based on assessments that are only linguistically equivalent could therefore be perceived through a hegemonic narrative instead of the emancipatory one that is being sought by the UN.

It is difficult to know the status of the paper on which I’m commenting (research or marketing). It describes a comprehensive and worthwhile exercise, but it will require comprehensive consultation and discourse among target countries to develop legitimate measures that are acceptable to all.

Reflections on Rosie Batty as Australian of the Year

As Rosie Batty’s term as Australian of the Year comes to an end, I would briefly like to reflect on the impact and possible lessons we could learn from her experience and advocacy.

Like most people, the first time I saw Rosie Batty on television was in an interview shortly after Luke’s death. That interview had a profound effect. It wasn’t filled with anger, hatred or blame. Instead it was filled with sadness, compassion and understanding; including towards Luke’s father.  That she could have been anyone’s mum, daughter or friend, made me suddenly realize that this sort of domestic violence could happen to anyone.

Each time I saw Rosie on television I thought that in a better world we would never have known her.   As Australian of the Year, each one of us would have gained their own insights and inspiration from Rosie’s experience. Mine, mundane as it is, is that we should all make an effort to ensure we never put anyone in Rosie Batty’s position again.

Throughout the reporting a few key things stood out for me. The incident was not random and the justice and welfare systems had sufficient interventions to identify the problem. Further, Luke’s father had a number of active arrest warrants and intervention orders. The police even had the opportunity to arrest and detain him, but did not do so because of problems with the police database.

Problems with the Victorian police database go back a number of years.  In 2005, the director of Police Integrity, George Brouwer, called for the database to be replaced and that cost should not be a deterrent. Assistant Commissioner Kieran Walshe at time did not agree and did not consider it a priority. Successive Victorian governments have history of problems in public sector IT services includeing CenITex, LEAP, MYKI and Ultranet to name a few.

Large computer systems are not hard or impossible by their nature. What makes these projects hard are greed and fanatical desires for efficiency.  What I’ve learnt from Rosie Batty is that things that ensure the safety and welfare of our children run deep, and each of us can make a difference at every level of society.  To make a better world we could begin with a duty of care when developing IT systems, get that right and the balance sheet will look after itself.

Sources

The Age – a report on the inquest

7.30 Report story

ABC Report – 2005, on LEAP database

CenITex story

The Uncanny Progressive versus Traditional Debate

Meditation on the following blogs

Dr Beardface  On reading (part 1)  On defending shit work  On ideology

Linda J. Graham  On tax-payer funded research   Angry white men

Greg Ashman   Come, join the enlightenment    Loose ends    The disconnect

debsnet   Traditional Progressivity or Progressive Traditionalism: Ditch the dichotomy

Corinne Campbell  On TeachMeets, EduChats and Marketing

My recent twitter feed has had much discussion about traditional and enlightenment values. While some seem satisfied with their respective positions, to me it’s a manifestation of an underlying discontent that’s been brewing throughout my 30 career in education, and these blogs provide an opportunity to consider these issues propelled by real people with real emotions, not abstract ones.

While the debate had material for many tangential excursions, I will restrict this blog to a couple of key themes – post structuralism, sex and race, the Enlightenment, and the role of teachers in a post traditional landscape. Further, I haven’t addressed all the blogs related to this debate.

This blog is part of my public thinking for my PhD, and the references are as much for my research purposes as for any academic pretensions.  This topic is really too big for twitter and the blogosphere so this is more of an essay, in some ways proving the point that the topic is too dense and complex.

Post Structuralism

Issues with post-structuralism in education drove much of the twitter debate to which I’m responding, perhaps it’s best to quickly summarise my understanding of this endeavour. Structuralism was an attempt to identify underlying structures, codes and conventions that produce meaning and make meaning possible.  However early structuralists like Barthes, Lacan and Foucault recognized that meaning making is not independent of the person making the meaning; that is, a subject’s sex, social class and ethnic identity affect meaning making.  This led to post structuralism and in particular Deconstruction led by Derrida who critiqued hierarchical oppositions in Western thought. Derrida showed that notions such as inside/outside, mind/body, nature/culture were not natural but a construction.  While the work of Deconstruction sought to dismantle and reinscribe textual meaning, it did not seek to destroy meaning. However, in effect, Deconstruction did become a teasing out of warring forces of signification within a text and is therefore associated with broader movements such feminist theory, various psychoanalytic theories, Marxist thought, Post-Colonial Theory, and Minority discourses (Culler, 1997, pp. 125–131)

Posts-structuralism has generated much academic activity and material, and even if a small percentage of this material is dross, this seems to be sufficient to attract much ridicule from traditionalists. Nevertheless, post-structuralism remains a valid and useful endeavour, particularly for education which has a key interest in matters of content and representation. Drawing on the notion of education’s instructional core(Elmore, 1996), students, teachers and content are the three central concerns of education; from a post-structural perspective this translates into two meaning-making subjects and a collection of externally produced content signifiers.  It is the concern with signification and the subject that makes post-structuralism particularly relevant to education, more so than some of the other ‘posts’ related to economics, management, art and culture (e.g. Drucker, 1993; Jameson, 1991). Furthermore, all these ‘posts’ sits within broader changes within western societies sometimes described as a condition of postmodernity (e.g. Harvey, 1990; Lyotard, 1984).

Post-structuralism has failed in many respects to live up to its political promise, while it provides a range of social enquiries it seemed to have had little interest in concrete political issues such as justice, freedom, truth and autonomy (Eagleton, 2008, p. 199). One example is post-structuralism’s scepticism of Government  (see Governmentality e.g. Foucault, Burchell, Gordon, & Miller, 1991; Foucault, 2008). Without recourse to an effective government it becomes difficult to mount a case for emancipatory interests such as equality. A case for equality requires a government capable of both monitoring equality and implementing effective policy in response. Equality requires ‘governmentality’, and universal education is traditionally provided by government. So post-structuralism’s scepticism and critique of the role of government has, unwittingly or otherwise, weakened the position of the state to define, monitor and redress disadvantage in education. Furthermore, in diminishing the state’s role in defining and redressing disadvantage, post-structuralism has, again perhaps unwittingly, opened up the landscape for market forces to redefine and address perceived disadvantage.  Post-structuralism can redress this by either better scoping out its concerns to focus on signification, or by developing a stronger narrative in favour of systems and government.

Irrespective, post-structuralism will continue to have a strong role to play in education due to its concern for signification and representation, particularly when it’s able to take a ‘structuralist’ stance to inform how subject matter should be represented in the digital age. This will continue to be a highly contested area (Beavis, 2010; Kress, 2003). For this reason, post-structuralism is unlikely to be usurped by the more contemporary post-humanism  (Barad, 2003) within the field of education any time soon.

Sex, Race and Uncanny Australia

The trad-prog debate also involved sex and race through the invocation of Angry-White-Men, a reformulation of the post-modern Dead-White-Men. This invocation generated some offence as well as ironic amusement.  There is no doubting the phenomenon of the violent angry male, but there are also men who are angry about other things such as Australia’s treatment of asylum seekers and quality of universal education.  Conflating these forms of anger may not be useful.

Collins Street

Collins St, 5p.m.1955, John Brack © National Gallery of Victoria 

The distinct strata that once divided men and women has evaporated  

The distinct strata that once divided men and women in Australia have also evaporated.  Two of the protagonists propelling the twitter exchange, for example, included a senior female academic and a male student, a reversal of traditional power relations.  These inversions are no longer isolated, Australia’s richest person is now a woman, and we have had a woman prime minister. Further, for each Alan Jones and Andrew Bolt in the public sphere there are is now a corresponding – and arguably more articulate and successful – Clementine Ford and Jane Caro.  Nevertheless, while ‘social media’ power between men and women may have equalised in the public sphere, this has not necessarily translated into real economic or political equality.

Similar issues exist for race. White Australia once was able offer generosity to its Asian neighbours for failed colonisation practices in Vietnam for example. In education this led to a culture of inclusiveness. White Australia is no longer able to assert itself through either colonisation or generosity in the same way from a position of power. Chinese nationals now have the economic upper hand to purchase property within the catchment areas of some of Australia’s most sought after public schools (Chinese buyers flock to Glen Waverley). The Australian economy is no longer able to assert itself within an Occidental context of dominant white Anglo-Saxon men and women. Australia is becoming increasingly dependent on Oriental forces (see Said, 1994 for post-colonial framing of Oriental and Occidental).

So the traditional framings of feminism and post-colonialism are no longer able to provide a coherent narrative of power relations in Australia in a way that resonates with the lived experience of many Australians. Most Australians, particularly in education, now routinely report to by both men and women of both Occidental and Oriental backgrounds.  This is not to say that there are no systemic structural inequities based on sex or race, and that sex and race are no longer valid targets of public policy, but structural inequities can no longer be fully explained in terms of hegemonic white male power.   Nor can white male anger be dismissed as a contemporary manifestation of dead white men, it is likely to be more pernicious than that and involve female protagonists (e.g. Pauline Hanson)

There is therefore unfamiliarity and strangeness around the roles of sex and race in power relations, boundaries that once distinguished one from the other may no longer be tenable or recognisable.  Gelder and Jacobs, drawing on Freud and Kristeva, developed the notion of an uncanny Australia with respect to the sacredness in Aboriginal culture(Gelder & Jacobs, 1998, p. 26).  This notion of uncanniness could be extended to sex and race, an uncanniness that could itself be the root of anger.

A flight to Enlightenment

A flight to Enlightenment and towards the certainty of empiricism is one response to an uncanny Australia and a more complex environment.  While such a flight could be dismissed as a simple psychological defence, it also seems part of a broader trend and therefore worthy of exploration.  For example, Geoff Masters, CEO of Australia’s preeminent educational research organisation, considers the field of educational assessment as currently divided and in disarray due to fault lines occurring between competing philosophies, methods and approaches (Masters, 2013, p. 1). As I have argued elsewhere, Masters’ response to this disarray is a unifying principle that takes a Kantian metaphysical philosophical stance, or an early Enlightenment stance. A stance that presupposes a cognition (presumably white male) before another cognition that acts as a philosophical arbiter of practical reason, judgement, and theoretical reason (Habermas, 1996, p. 2).  Masters’ proposed principle also privileges the role of objective measurement and the Rasch Model (Masters, 1982) of which Masters is a world leading exponent. In doing so Masters also regresses to an early version of the Enlightenment that ignores Hegel’s work in showing that philosophy is not transcendental but historically located (Singer, 2001, p. 13)

Furthermore, education at heart is not a science but a social activity. Education does deal with facts, but mainly deals with norms that are socially constructed.  Facts and Norms should not be confused. Curriculum, for example, cannot be determined by empirical means. Instead, curriculum is developed by drawing on social norms and social reasoning and articulates the shared expectations of a broader community.  Even for those aspects of education that can be measured, the notion of causation is less well understood,  and the validity of meaning making is underdeveloped and under-theorized (Markus & Borsboom, 2013, p. 15). Blind experiments that are able to test some of the more contentious issues are also not possible in education due to ethical constraints, so for many of these issues effective social reasoning is required because empiricism is simply not an option.

While a retreat to the Enlightenment may be comforting and provide certainty in a time of uncanniness, even those dedicated to retrieving the Enlightenment, such as Habermas, emphasise the centrality of moral discourse and pragmatics (Habermas, 1985, 1987, 1996, 1998).

Teachers and Systems

From an effective teachers point of view the dichotomy between traditional and progressive, or any hierarchical oppositions, make little sense. Teachers are practical and pragmatic reasoners who, when given sufficient autonomy and support, use their educational expertise, their engagement with the broader educational community, and their knowledge of their students, to deliver lessons that effortlessly traverse oppositions. It is this skill and experience that makes teachers excellent social reasoners and moral agents. However articulating these skills with systems remains problematic.

Systems provide the resources and administrative authority for teachers to conduct their work, and the system-teacher relationship requires reciprocity.  Where this reciprocity is distorted it can lead to systems colonizing the world of teachers (see Habermas, 1987). One example of where the nature of reciprocity has changed relates to educational standards. Traditionally teachers, as moral agents, contributed significantly to standard setting exercises that reflected social expectations (Cizek, 2012). However, the social process of standard setting is increasingly being replaced by instrumentally defined cut-off points and levels (e.g. see OECD, 2012, pp. 258–263) which may be appropriate for system evaluation but perhaps less so for reporting to students and parents.  The diminishing role of subject associations is evidence of this transition which has led to a weakened relationship between teachers and systems.  System consultations with the teaching profession are being increasingly replaced by private discussions among board-level coteries. There is also the phenomenon of teachers in leadership positions being appropriated (bought, seduced, corrupted) by commercial interests.

So while there is reason to be sanguine about the capacity of teachers to navigate divides within their classrooms, systemic problems remain that require political action to generalise teacher experience across systems.

Looking Ahead

Julia Gillard prime ministership may provide a useful glimpse of what the future might look like. Germaine Greer describes it thus

it’s important to realise that Julia Gillard is part of a coalition. What that means is that she has to negotiate every single policy position. What that means is camel trading on the floor. It happens to be what she’s good at. You can say, ‘We want to know what she really, really believes.’ In fact, it’s irrelevant because whatever she really, really believes is not what’s going to happen.(“Q&A :Politics and porn in a post-feminist world,” 2012)

To me, this is the future. It doesn’t matter what any of us think, it is our capacity to engage and negotiate issues into action that makes us effective.  Who knows what the world would look like when men and women are equal, where the Occident and the Orient are equal, and where the Palestinian-Israeli conflict is resolved.  Nobody knows, and we will only find out if we take the necessary steps forward to engage and negotiate. There is a strong argument to be made that Gillard has been Australia’s most effective Prime Minister, not by way of being able to unify a majority around a single set of ideas in the manner of Bob Hawke, but by way of being able to effectively negotiate difference in a manner that delivered a more enlightened post-traditional society.  Further, Gillard’s post-traditional effectiveness was matched by a traditional hostility including that of Greer, who quickly followed up the above quote with comment that Gillard had a ‘big arse’; probably one of the most disappointing moments in Australia’s gender debate. Almost uncanny.

Barad, K. (2003). Posthumanist performativity : Toward an understanding of how matter comes to matter. Signs, 28(3), 801–831.

Beavis, C. A. (2010). English in the Digital Age: Making English Digital. English in Australia, 45(2), 21–30. Retrieved from http://www98.griffith.edu.au/dspace/handle/10072/37149

Cizek, G. J. (Ed.). (2012). Setting Performance Standards : Foundations, Methods, and Innovations. New York: Routledge.

Culler, J. (1997). Literary Theory: A Very Short Introduction. Oxford: Oxford University Press, UK.

Drucker, P. (1993). Post-Capitalist Society. Routledge.

Eagleton, T. (2008). Literary Theory : An Introduction, Anniversary Edition. Minneapolis: University of Minnesota Press.

Elmore, R. F. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1), 1–26.

Foucault, M. (2008). The Birth of Biopolitics: Lectures at the Collège de France, 1978–1979. (G. Burchell, Trans., A. I. Davidson, Ed.). New York: Palgrave Macmillan.

Foucault, M., Burchell, G., Gordon, C., & Miller, P. (1991). The Foucault Effect: Studies in Governmentality. Chicago: University of Chicago Press.

Gelder, K., & Jacobs, J. M. (1998). Uncanny Australia: Sacredness and Identity in a Postcolonial Nation. Carlton South: Melbourne University Press.

Habermas, J. (1985). The Theory of Communicative Action: Reason and the rationalization of society. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1987). Lifeworld and system: a critique of functionalist reason. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1996). Moral Consciousness and Communicative Action. (C. Lenhardt & S. W. Nicholsen, Trans.). Cambridge MA: MIT Press.

Habermas, J. (1998). Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. (W. Rehg, Trans.). Cambridge MA: MIT Press.

Harvey, D. (1990). The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change. Cambridge MA: Blackwell.

Jameson, F. (1991). Postmodernism, Or, The Cultural Logic of Late Capitalism. Duke University Press.

Kress, G. (2003). Literacy in the New Media Age. London: Routledge.

Lyotard, J.-F. (1984). The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press.

Markus, K. A., & Borsboom, D. (2013). Frontiers of Test Validity Theory : Measurement, Causation, and Meaning. New York: Routledge.

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. doi:10.1007/BF02296272

Masters, G. N. (2013). Reforming Educational Assessment: Imperatives, principles and challenges. Australian Education Review. Retrieved from http://research.acer.edu.au/aer/12

OECD. (2012). PISA 2009 Technical Report. Paris: OECD Publishing. doi:10.1787/9789264167872-en

Q&A :Politics and porn in a post-feminist world. (2012). Australia: Australian Broadcasting Corporation. Retrieved from http://www.abc.net.au/tv/qanda/txt/s3451584.htm

Said, E. W. (1994). Orientalism. New York: Vintage Books.

Singer, P. (2001). Hegel: A Very Short Introduction. Oxford: OUP Oxford.

 

On Angry White Men

Relations between the sexes seem a lot more toxic now than I would have imagined when I was young.   During the sixties, the decade of my birth, the roles of men and women seemed to consist of distinct and persistent patterns that quickly evolved with the availability of the contraceptive pill and broader social movements. A convincing case for sexual equality was prosecuted throughout the seventies and as a teenager it pretty much seemed to me that the traditional male role of dominating nature had become redundant; bridges were easy, the Ford Falcon GTHO Phase 3 was the epitome in car production, and the world had enough bombs to destroy the earth several times over.  Man (sic) had reached the moon, and the next step to Mars seemed to involve solving social problems more than technical ones.  What was once considered a man’s traditional work had been done. So I entered the workforce quite prepared for a working life where gender roles would evolve significantly and where work was to become more socially and less technically focussed.  A grand narrative had been established, and Deborah Wardley’s victory against a silly Reg Ansett that allowed her to pilot a plane seemed a first step in a long trajectory of social progress.  Yet relations between the sexes seem more toxic now than then.

At the political level there have been mixed and sometimes troubling results. The highly admired Joan Kirner was education Minister and then State premier during the first years of my teaching career and her reforms to the VCE put Victoria in a very good place educationally.  At the time it seemed inevitable that Joan would be the first of many women premiers for Victoria, but over 20 years and 6 premiers later there has been no further progress.  Similarly for Western Australia, where there has been no progress since a concerted hatchet job was conducted against a very competent and intelligent Carmen Lawrence.  The current political environment continues to be toxic; women are welcomed as loyal deputies but shunned when they manifest any will to power.  There is the case of Julie Bishop, did she or didn’t she participate in a power play, while Truss’s subversion on Macfarlane is regarded as part of good sport.  Anthony Albanese continues to be regarded as good bloke having lost his leadership challenge, but had a woman contested and similarly lost it’s not hard to imagine that she might be characterised as spurned and brooding. How are women to be effectively socialised into leadership in such a toxic environment, and what effect does this role modelling have on relations between the sexes at work in general, school and daily life. My view is that history has not yet fully recognised the achievements of Julia Gillard, and the inevitability of  Australia having more women prime ministers in the near future is not so certain.

So here we are, things have moved significantly since the death of Barthes (1980) and Foucault (1984) yet things also seem more toxic. While the critiques of Barthes, Foucault and Derrida may still have relevance, the social conditions that they described no longer exist.  Women still encounter unfair structural and systemic hurdles to their expressions of identity and power, but these hurdles are no longer as universal and uniform as they once were. While women still encounter systemic disadvantages, there’s now also a sufficient critical mass of competent and powerful women to change the dynamic.   We now need to develop frameworks and approaches that remove toxicity from these swirling and evolving power relations.

Many men are angry but the reason for anger varies.  Some men are dangerously angry because they grieve a loss of control over women; lock them up. Some are angry because they can no longer use nature as their playground; educate them. Others are angry because objective instrumental reasoning is dead; these can be indulged a little and exposed as they’re no longer relevant. Yet others are angry because they consider humans behaving poorly towards each other and towards their environment; engage them for their energy. I draw my inspiration for anger and energy from the likes of Henry Rollins.

New theoretical positions need to be developed for the contemporary world. Foucault, Barthes and Derrida probably no longer cover it. Greer has inspired many but her contribution in calling out Julia Gillard for her ‘big arse’ has been less than useful and only contributes to toxicity.  From an educators perspective the work of Butler, Nodding and Gilligan continue to be informative.  My view is that there is further potential in integrating an ‘ethics of care’ within a grander narrative of justice. Gilligan and Kohlberg did work on this some time ago but it could be revisited.  Then there is Judy Wajcam’s work on technology and techno-feminism that challenges views towards technology.

Never mind the bickering, there’s work to be done.

The Demise of Teacher Professional Judgement

Follow up to Constellation or Continuum – metaphors for assessment

There are many ways in which teacher professional judgement can shape schooling.  Teachers can participate in the development of study designs, curriculum and syllabus, and they can also participate in exam setting, exam marking and standard setting.  In this way teachers perform sophisticated social roles in mediating between systems and the lifeworld of students as well as in setting and maintaining educational norms and expectations on behalf of the community. This kind of participation, where teachers both contribute to the creation of norms and learn how to teach them, is present in all systems to some extent, and highlights the important roles as moral agents and moral leaders that teachers can have.   However there are currently two developments working against teachers taking on system roles as moral agents:  1) instrumental reasoning of mathematical models and 2) the post-conventional/post-traditional nature of technology based education making teacher participation problematic.

Instrumental Reasoning

Where once curriculum and assessment were reflections of social expectation (including expectation of industry), this normative function has to some extent been superseded by uni-dimensional models of curriculum and assessment, mainly the Item Response Theory models (e.g. see Ayala, 2009; Embretson & Reise, 2000; Masters, 1982; Rasch, 1980) and its associated continuum metaphor.  In education systems where Item Response Theory models becomes prevalent learning progressions are less determined by social expectation and more determined by instrumentally defined scale progression, so that curriculum begins to comprise of ‘content that scales’ instead of content that meets social expectations.  Once curriculum content is comprised of ‘content that scales’, teachers’ participation in standard setting is no longer a requirement as instead of socially defined educational standards these standards can be set by way of cut-points, cut-scores and bands instrumentally and arbitrarily defined by application of Item Response Theory  based algorithms.

My thesis will argue that this phenomenon can lead to various outcomes including 1) alienation of teachers’ work, 2) curriculum and assessment not addressing social expectations, 3) students alienated from society and not fully socialised, and 4) a general loss of social capital across the system. It can also be seen as very efficient and cost saving as it doesn’t require expensive teacher engagement.

Post-conventional or post-traditional nature of education

The need to develop new educational norms and expectations during a time of developments in digital technology presents another issue for teacher engagement. Beavis (2010, p. 26) articulates this well when she states that factors such as cultural heritage and identity are at play for not only the student and teacher but also the subject itself.  The required moral reasoning of teachers is therefore far greater at a time where the system capacity of teachers has been greatly diminished through cutbacks etc. This leaves a vacated landscape that private sector can seek to fill (e.g Ultranet see Bajkowski, 2013), or other consortia (e.g. 21st Century Skills see Griffin, McGaw, & Care, 2012).

Discussion

Not all contemporary assessments are grounded on mathematical models. For example the Victorian Certificate of Education (VCE) is one example of curriculum and assessment that is firmly socially grounded.  The study designs for the VCE (VCE study Designs)  reflect the social, cultural and economic activity of Victoria, and Victorian teachers are actively involved in its design and implementation, including exam setting and marking. The VCE also uses routine statistical techniques (standardization and normalization) to create a single score and then ATAR for students that can be used as currency in the future job and education market in Victoria and beyond. These features make VCE a highly regarded qualification but that it has such significant social buy-in will make it difficult to adapt to technology-based. Although this can be overcome with good management, good planning and sufficient resources for stakeholder engagement.

There is also some hope produced by the constellation metaphor and in the use of Bayesian techniques in the development of curriculum and assessment that is more comprehensive (e.g. Almond, Mislevy, Steinberg, Yan, & Williamson, 2015). However the establishment of good Bayesian belief networks also requires extensive experienced teacher participation, so the danger of the constellation metaphor is that instead of relying on teachers’ input for belief networks, these networks will instead by based on trawling through learning analytic data. Should this occur, my thesis is that this would also lead to alienating circumstances for teachers and students.

My thesis will develop with the view that sophisticated and social cohesive education systems have a sufficient base of morally competent teachers that are involved in the setting of curriculum and assessment, where the judgement of these teachers are informed and supported by sophisticated data systems (constellation and continuum). Of course this could potentiality bifurcate the other way, where teachers and students become increasingly alienated by technocratic systems.

Almond, R. G., Mislevy, R. J., Steinberg, L., Yan, D., & Williamson, D. (2015). Bayesian Networks in Educational Assessment. Tallahassee: Springer.

Ayala, R. J. De. (2009). The Theory and Practice of Item Response Theory. Guilford Press.

Bajkowski, B. J. (2013). News Review . Vic Auditor fails Ultranet, (March).

Beavis, C. A. (2010). English in the Digital Age: Making English Digital. English in Australia, 45(2), 21–30. Retrieved from http://www98.griffith.edu.au/dspace/handle/10072/37149

Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. L. Erlbaum Associates.

Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and Teaching of 21st Century Skills. Dordrecht: Springer. doi:10.1007/978-94-007-2324-5

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. doi:10.1007/BF02296272

Rasch, G. (1980). Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: MESA PRESS.

Constellation or Continuum – metaphors for assessment

In this post I want to lay ground work for a major shift in assessment methodology that education will experience in the coming decades. It will do so by discussing educational objectives, heuristic metaphors, and mathematical models.

To be clear, what we are talking about here are mathematical models and how they implement metaphors or ways of verbal reasoning about educational objectives.  These models inform how we think about and organise content, including assessment, at the system level. While this post will remain agnostic on the science of how the brain works, these models nevertheless inform how we approach students and organise schooling.

After discussing two metaphors, this post will discuss potential issues in their use with a view to informing teacher participation in a broader debate. While it may be unreasonable to expect teachers to understand the mathematics, it is reasonable to expect teachers to engage at the metaphorical and verbal levels.

Constellation or Continuum

The constellation and continuum metaphors have long and evolved histories in the academic and published literature. I will discuss these metaphors in terms of their main exponents and uses.

The continuum metaphor, or the ruler metaphor, is the one most Australians would be familiar with or have experienced.  It is the metaphor used by both NAPLAN and PISA as part of system evaluation. It is therefore also used in many derivative studies or by those who wish to align themselves with these methodologies.  Australia has many world leading exponents for the continuum metaphor with Geoff Masters the most well know due to his development of the Partial Credit Model (Masters, 1982), which was a development of the earlier Rasch Model (Rasch, 1980).  The mathematical models associated with this metaphor are generally called Rasch Models or Item Response Theory (e.g. see Ayala, 2009; Embretson & Reise, 2000) which are often described in terms of improvements to Classical Test Theory.

The constellation metaphor is not so well known in large scale assessment.  A well know exponent is Robert Mislevy who, while remaining pluralistic, opened up the field through his work with others in Evidence Centred Design (ECD) (Almond, Mislevy, Steinberg, Yan, & Williamson, 2015; Mislevy, Steinberg, Almond, Haertel, & Penuel, 2003). This metaphor can also be associated with diagnostic assessment or cognitive assessment (e.g. Leighton & Gierl, 2007, 2011; Rupp & Templin, 2008). The mathematical models associated with this metaphor include Bayesian Networks, Neural Networks and elaborations of Item Response Theory. The constellation metaphor is not as widely used as they are more difficult to implement, although they are often used in post-hoc analysis of learning data.

A simple example

The profound differences between the two metaphors can be illustrated through a simple example. Below is a diagram showing a simple test of 8 questions which tests four operations using smaller numbers then larger numbers.  Student A can do all operations but not with larger numbers. Student B can just do addition and subtraction.

Responses

The key issue here is that each student has quite a different state of proficiency yet the raw score for these two patterns cannot distinguish between them, so raw scores mathematical models as used by the continuum metaphor cannot readily detect this type of difference.  A deviant response pattern may be picked up in a misfit or bias analysis, but unless there is some additional treatment these two students will be reported the same.

The two ways of reporting these two response patterns under each metaphor is illustrated below.

stars_ruler

It is clear that differences between the two students are lost under the continuum metaphor, but are captured under the constellation metaphor.

My hypothesis is that Australia is captured by the continuum metaphor due to the good fortune of it having the leading Item Response Theorists in the world (Masters, Adams, Andrich, Wu, Wilson etc), it is this circumstance that has also led to a neglect of the constellation metaphor and a concern about what individual Australian students are able to do; a neglect that has led to a decline in overall student performance and to a paradoxical situation where Australia is well placed to measure its decline.  This is a hypothesis only that cannot be empirically proved but which can be reasoned about.

Furthermore, I also contend that the continuum metaphor, with its focus on measurement, comparability and comparisons, is sometimes mistaken for neoliberal forces. It’s not really a conspiracy, but just a by-product of some smart people working very effectively in the endeavor of their interest.

Discussion

The constellation and continuum metaphors have corresponding metaphors for how we talk about teaching.  Related to constellation metaphor is ‘who a student is’, ‘collection of knowledge’, ‘learning as growth’ and ‘depth and relation’. Related to the continuum metaphor is ‘where a student is’, ‘uni-dimensionality’, ‘teacher as conduit’, ‘learning as filling an empty vessel’.

A particularly effective use of the continuum metaphor is as a system evaluation tool, that’s why it’s used in PISA, NAPLAN and TIMSS.  As a system evaluation metaphor it is also very effective at detecting system biases and therefore it served both accountability and civil rights movements in the United States during last century (see Gordon, 2013), which in part has led to the dominance of the metaphor today.

What is clear from the example above is that the continuum metaphor, and by extension NAPLAN, is a poor diagnostic device and is able to provide little information about the student and on what to teach next, other than a vague location where a student may be in relation to other students.

While the constellation metaphor is better at providing diagnostic information to teachers, these sorts of assessments are also a lot more difficult to manage and implement and have therefore not been implemented at scale. Instead, the constellation metaphor is increasingly being used for post-hoc analysis and fishing exercises on causal relations in education; for example learning analytics (e.g. Behrens & DiCerbo, 2014).  For those who consider education as a purposeful activity, this type of post-hoc meaning making may be of concern.

I trust this may help some, writing it has helped clarify some of my thoughts.

Addendum

Where both the constellation and continuum metaphors are driven by mathematical models, the determination of matters such as bands and cut-scores are largely arbitrary and determined by a choice of parameter. This contrasts to traditional standard setting procedures that are based on the professional judgements of groups of teachers (e.g. see Cizek, 2012) or holistic judgements in higher education (e.g. see Sadler, 2009).  The metaphors can of course be used to support teacher judgement, and some methods in Cizek’s book recommend this.

Almond, R. G., Mislevy, R. J., Steinberg, L., Yan, D., & Williamson, D. (2015). Bayesian Networks in Educational Assessment. Tallahassee: Springer.

Ayala, R. J. De. (2009). The Theory and Practice of Item Response Theory. Guilford Press.

Behrens, J. T., & DiCerbo, K. E. (2014). Harnessing the Currents of the Digital Ocean. In J. A. Larusson & B. White (Eds.), Learning Analytics:From Research to Practice (pp. 39–60). New York: Springer.

Cizek, G. J. (Ed.). (2012). Setting Performance Standards : Foundations, Methods, and Innovations. New York: Routledge.

Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. L. Erlbaum Associates.

Gordon, E. W. (Ed.). (2013). To Assess, to Teach, to Learn: A Vision for the Future of Assessment : Technical Report. Retrieved from http://www.gordoncommission.org/rsc/pdfs/gordon_commission_technical_report.pdf

Leighton, J. P., & Gierl, M. J. (2007). Cognitive Diagnostic Assessment for Education: Theory and Applications. New York: Cambridge University Press.

Leighton, J. P., & Gierl, M. J. (2011). The Learning Sciences in Educational Assessment: The Role of Cognitive Models. Cambridge University Press.

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. doi:10.1007/BF02296272

Mislevy, R. J., Steinberg, L. S., Almond, R. G., Haertel, G. D., & Penuel, W. R. (2003). Leverage points for improving educational assessment (PADI technical report 2). Menlo Park: SRI International.

Rasch, G. (1980). Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: MESA PRESS.

Rupp, A. A., & Templin, J. L. (2008). Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art. Measurement: Interdisciplinary Research & Perspective, 6(4), 219–262. doi:10.1080/15366360802490866

Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education.

 

Are Post-Structuralists too Dense and Complex?

Here’s a quick rejoinder to the blogging conversation between Greg Thompson, Greg Ashman and Naomi Barnes.

To recap, the initial Thompson post celebrated a couple of books on technology and culture and was explicitly unapologetic for the use of big words.  Ashman rejoined by questioning the value of certain types of scholarship, particularly various traditions of sociology that use big words, concluding that some sociological traditions need to do a better job of explaining themselves.  Barnes then rejoined from a post-structural, feminist, critical race perspective; saying good writing is important but this is hard in a genre dominated by white men of the Enlightenment.   While the arguments were well addressed in the various blogs, I feel the urgency of the underlying issue was not fully appreciated and I will attempt to redress that a little here.

WhileBlokes

white men

To begin, whatever the term, Occidental culture has experienced a seismic latent cultural shift since the sixties, a shift signified by many events. These events, including the 1967 summer of love in San Francisco, the 1968 Paris riots, the 1972 demolition of the Pruitt Igoe estate in St Louis, the 1973 oil crisis, the release of Koyaanisqatsi,  the art of Jeff Koons, among a range of other events, led to what some characterise as a condition of postmodernity (e.g. Harvey, 1990; Lyotard, 1984), a condition that needed to be described and theorised leading to various theoretical positions including postmodernism and post-structuralism.  These are in turn associated with the linguistic turn in philosophy, which focussed on the role of language in creating reality.

The decades of the linguistic turn of course produced much dross, and it would not be hard to find examples to sustain Ashman’s argument of too ‘Dense and Complex’.  Nevertheless such examples should not discount the worthiness of the endeavours to theorise the contemporary world. However post structuralism is not beyond criticism, skepticism or challenge. Eagleton (2008, pp. 199–200) for example, in an afterword to his 25th anniversary edition on literary theory, observes that as the 1980s wore on that post-structuralism failed to deliver on its political promise, with the German tradition including Habermas –  tenaciously clinging to topics such as discourse, justice, autonomy and ethics –  being better placed to provide a response to circumstances.  More recently others and in different traditions, such as Barad (2003), consider language as having been given too much power due to the linguistic turn and propose an alternative in posthumanism .  Given the coherence of Barad’s position, Ashman may find relief from dense and complex word smithing in her work.

The dichotomy between positivism and post-structuralism implicit in the blogs concerned me. This dichotomy does not seem appropriate for educational research or social science in general.  For example, while much quantitative educational research is dense with numerical methodology, all educational research and assessments are underpinned by educational content that is represented in some way.  It is within the brief exchange of signifiers between assessor (stimulus) and assessee (response) that post structuralism, with its tradition of addressing representation, has a role to play.  This is particularly the case as technology is restructuring the field of representation and communication leading to massive power struggles (Kress, 2003).   One current example is the big shift in how team work is being represented in schools, what used to be team sport, team dance, and team music is now appropriated thought proprietary technology (e.g. Griffin, McGaw, & Care, 2012). It is here that post-structuralists could insinuate themselves in the creation of new representational forms to be used within education to ensure that feminist and post-colonial concerns, as well as a bag of others, are addressed from the outset in the introduction of technology.  Rather than critiquing commercial implementations post-hoc, there remains an opportunity to influence given that many high stakes exams remain predominantly paper-based due to, from my perspective, a failure to agree on matters of subject matter representation using new technologies.  Post structuralist could definitely do more in asserting and explaining themselves here.

However, the real issue around which I would like to generate a sense of urgency is at the heart of Thompson’s initial post, the industrialization of time that sees humans being reconfigured for a fragmented world with disconnected events.  These observations resonate with my lived experience, amplify my own reading (e. g. Wajcman, 2015), and make me very concerned for future generations. It is towards addressing these issues, not just to identify them, that post-structuralists in education have their future work cut out.

Barad, K. (2003). Posthumanist performativity : Toward an understanding of how matter comes to matter. Signs, 28(3), 801–831.

Eagleton, T. (2008). Literary Theory : An Introduction, Anniversary Edition. Minneapolis: University of Minnesota Press.

Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and Teaching of 21st Century Skills. Dordrecht: Springer. doi:10.1007/978-94-007-2324-5

Harvey, D. (1990). The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change. Cambridge MA: Blackwell.

Kress, G. (2003). Literacy in the New Media Age. London: Routledge.

Lyotard, J.-F. (1984). The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press.

Wajcman, J. (2015). Pressed for Time: The Acceleration of Life in Digital Capitalism. Chicago: University of Chicago Press.

The possibility of a unifying principle for assessment

[thank you to all those supporting me to date in much greater numbers than I had expected. It’s a bit difficult to stick with my longish blogs. I’m sharing my pre-confirmation PhD thinking for today so apologies for the dryness and density, but I feel that we need to go here to engage the neoliberal agenda, lighter material to come later]

Thought piece on Geoff N. Masters – Reforming Educational Assessment: Imperatives, principles and challenges

ConstelationRuler_1

which metaphor for an assessment principle – constellation or continuum?

It is easy to agree with Geoff Masters (2013, p. 1) when he observes educational assessment as a field divided and in disarray.  Educational assessment began by providing simple reliable indicators to parents as well as to students for currency in the job and education markets.  Assessment has now grown to encompass school and system evaluation as well as scientific research, with elements of quality management and market research creeping in. Data collection is moving from research as event to embedded and ongoing research through ubiquitous and unobtrusive data collection  (Behrens & DiCerbo, 2014). This transition is blurring the demarcation between educational assessment and other forms of data collection.

While it’s easy to agree on the disarray, Masters’ unifying principle to address the chaos is problematic. Masters proposes that the fundamental purpose of assessment is to establish where learners are in their learning at the time of assessment (2013, p. 5), but this principle seems too attached to the objective measurement school and its philosophical stance.

The problem that Masters is sensibly trying to address is the divided approaches and paradigms in contemporary assessment practices such as quantitative, qualitative, formative, summative and the like. Masters addresses this problem by suggesting a universal transcendent principle to underwrite all assessment practices. But for his principle to be unifying, universal and useful it must be better than competing alternate ways of formulating a principle.  So is Masters’ principle something we could all agree to over other contenders for universal principles? While I do not propose to proffer an alternative at this stage, let’s explore Masters’ principle a little further.

Masters’ unifying principle is presaged by a learning space, either unidimensional (continuum) or multidimensional (continua), in which a student can be located at a particular point in time.  The language of the principle is about mathematical space and location, and by incorporating this metaphor into a principle he seeks to subsume all assessment practices.  His principle assumes that there is a true location at which each learner can be located at a point in time, and that once that location is determined that information can be used to fulfil all possible educational information purposes.   So there are two issues, is the location metaphor the best way to describe contemporary assessment practices, and is a location – should it be able to be determined – once determined be sufficient to meet all educational information needs.

The foundation for Masters’ principle appears to be the objective school of measurement with its Rasch-based and IRT-based models (e.g. see Embretson & Reise, 2000; Masters, 1982; Rasch, 1980). It is this school of measurement with its concerns for true score and measurement error that lends itself to the ‘where is the student’ metaphor.   However, there are increasing calls for the use of other measurement models for which the ‘who is the student’ metaphor is probably more appropriate. Notable examples of this work includes that of Mislevy as well as that of Leighton and Gierl (Almond, Mislevy, Steinberg, Yan, & Williamson, 2015; Leighton, Gierl, & Hunka, 2004; Leighton & Gierl, 2007, 2011). These alternative models, by moving away from the singular location metaphor, challenge the usefulness of Masters’ unifying principle.

There are several ways of describing and locating Masters’ unifying principle.  One that comes to mind is that Masters takes a Kantian approach with its focus on objective transcendence presupposing learning as moving from location to location. From the objective measurement school this is couched as ‘the idea of the variable must transcend any particular set of observations and the measure on the  variable must transcend the observed responses on which it is based’ (Wright & Stone, 1979, p. 141), where what is learning is seen as an a priori concept measured by the subject through empirical observation; along with appropriate application of measurement error.  By casting Masters’ approach as Kantian allows us to quickly sketch out a landscape of alternative foundations for a unifying principle.

Unlike Kant, Hegel took history into account. Where Kant thought he could say on purely philosophical grounds what human nature is and always must be, Hegel accepted that the Human condition could change from one historical era to another (Singer, 2001, p. 13). The Hegelian notion of a dynamic history challenges the stability of Masters’ notion of ‘establish where learners are’, because this location is dependent on historical context.  It’s then a fairly short leap to a Marxist critique of the principle, that any measure used to implement the principle could be biased against certain groups which of course could be mitigated by techniques such as DIF.  It is at this point that I find we can discard Masters’ principle from being universal, and that it’s at best a useful heuristic. This brief analysis points to the danger of basing principles on an instrumental technique, in this case the Rasch model. A principle should probably come before selecting a technical implementation.

When considering assessment from a Marxist perspective, and within the context of Lyotard’s (1984) analysis of knowledge , three further approaches become apparent. The first one is neo-liberalism and its concern for performativity (Ball, 2003) which Lyotard (1984, p. 54) describes as being defined by an input/output ratio. Masters’ Rasch model provides a particular advantage here over other models such as Bayesian networks .  As Masters has earlier stated, in order to enable quantitative comparisons, or make ratios, we need a linear scale that makes differences between persons the same wether through hard or easy items(Wright & Masters, 1982, p. 8). That is, the Rasch model’s ability to create linear scales dovetails neatly into neoliberalism’s need for ratios. Masters may therefore be inadvertently buttressing a neoliberal agenda with his unifying principle.

Returning to Lyotard(1984), the Marxist agenda bifurcated around the time his book was published into what I characterise as post-structuralists and neo-modernists. On assessment, the post-structuralist due to their incredulity of grand-narratives (in particular those that involve numbers) continue to take a suspicious stance towards systems and system assessment.  This stance has continued to grow since early days of the Frankfurt school in particular Marcuse and his notion of the Great Refusal (Marcuse, 1974, 2012). Post-structuralists therefore can find it difficult to engage with system assessment in a positive sense, but they have a lot to say about the lives of individuals within the lifeworld which continues to be valuable for system assessment.  Neo-modernists on the other hand, in the tradition of Habermas (1985, 1987), are simpatico with the petit narratives of the post-structuralists but engage more constructively with systems. Neo-modernists consider the system to have emancipatory potential while having a tendency to colonize the lifeworld of communities that needs to watched and mitigated through transparency and deliberate democratic processes. From a neo-modernist perspective, a principle should be based around what is sought to be achieved, what needs to be understood, or what needs to be coordinated across the system. A neo-modernist will continue to embrace the objective measurement school strongly however, because of objective measurement has a strong ability to determine DIF, bias, and fairness. But objective measurement would not presage a universal principle on assessment.

This author will continue to work in a modernist tradition towards one or more universal principles for assessment to provide alternative to Masters which I consider too close to the neoliberal agenda.

Almond, R. G., Mislevy, R. J., Steinberg, L., Yan, D., & Williamson, D. (2015). Bayesian Networks in Educational Assessment. Tallahassee: Springer.

Ball, S. J. (2003). The teacher’s soul and the terrors of performativity. Journal of Education Policy, 18(2), 215–228.

Behrens, J. T., & DiCerbo, K. E. (2014). Harnessing the Currents of the Digital Ocean. In J. A. Larusson & B. White (Eds.), Learning Analytics:From Research to Practice (pp. 39–60). New York: Springer.

Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. L. Erlbaum Associates.

Habermas, J. (1985). The Theory of Communicative Action: Reason and the rationalization of society. (T. McCarthy, Trans.). Boston: Beacon Press.

Habermas, J. (1987). Lifeworld and system: a critique of functionalist reason. (T. McCarthy, Trans.). Boston: Beacon Press.

Leighton, J. P., & Gierl, M. J. (2007). Cognitive Diagnostic Assessment for Education: Theory and Applications. New York: Cambridge University Press.

Leighton, J. P., & Gierl, M. J. (2011). The Learning Sciences in Educational Assessment: The Role of Cognitive Models. Cambridge University Press.

Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach. Journal of Educational Measurement, 41(3), 205–237. doi:10.1111/j.1745-3984.2004.tb01163.x

Lyotard, J.-F. (1984). The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press.

Marcuse, H. (1974). Eros and Civilization: A Philosophical Inquiry Into Freud. Beacon.

Marcuse, H. (2012). One-Dimensional Man: Studies in the Ideology of Advanced Industrial Society (Vol. 8). Beacon Press.

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. doi:10.1007/BF02296272

Masters, G. N. (2013). Reforming Educational Assessment: Imperatives, principles and challenges. Australian Education Review. Retrieved from http://research.acer.edu.au/aer/12

Rasch, G. (1980). Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: MESA PRESS.

Singer, P. (2001). Hegel: A Very Short Introduction. Oxford: OUP Oxford.

Wright, B. D., & Masters, G. N. (1982). Rating Scale Analysis. Chicago: MESA PRESS.

Wright, B. D., & Stone, M. H. (1979). Best Test Design. Chicago: MESA PRESS.