Critical English Online
Assessment in the primary school
Prior to the establishment of a national compulsory system of primary education in 1877, assessment practices in colonial schools, at the time administered by the young country's provincial governments, were knowledge-centred and at times quaint. (Check out Mary Hill's Historical overview for more detail on the development of assessment practices up until the 1990s.)
The language of standards enters the equation particularly strongly after 1877, when national standards were set for primary students who, after their initial years of schooling, were actually put into classes called Standard 1, Standard 2, and so on until Standard 6 (now Year 8) when pupils sat a national "Proficiency" examination. Thus, in Standard 1 for example, a student would be expected to achieve the following "Reading" standard: Sentences composed of words of one syllable, and common words of two syllables, to be read intelligently.
Unsurprisingly, teaching to the test became a predictable outcome of the regime, and it was not uncommon for students to learn, by rote, prescribed books. In the 1920s and 30s, however, progressive discourses centred on the "natural" development of the child were beginning to challenge the ideology of national, uniform, prescribed standards. However, these discourses tended to co-exist, rather uneasily, with standardised testing regimes that purported to measure children in terms of externally referenced norms.
While the latter did not disappear, between the 1950s and the 1980s the emphasis shifted towards more descriptive, "child-centred" and diagnostic approaches to assessment, with a focus on individual student capability and how it might be fostered. The pendulum was swinging away from nationally prescribed standards (or measurement norms) towards attempts to systematically record student progress through such devices as running records and writing conferences.
As a result, teachers led the movement to replace norm-referenced judgements (for example, recording progress with 1-5s) with descriptive, criterion-referenced comments. This was consistent with notions of child-centredness, constructivist learning theories and the teachers' desire to work with children individually and in groups to improve their achievement.
It wasn't until after the restructuring of education in the 1990s that accountability discourses based around marketisation, managerialism and "new professionalism" re-established themselves, and "national standards" being repopularised throughout the Western World started struggling anew for dominance. Major drivers in the reintroduction of national standards included: making schools self-managing and individually responsible for reporting their students' achievement of standards in 1989 and the introduction of a new levels-based national curriculum in 1993.
As a result of the changes to school administration and the reorganization of the curriculum into achievement objectives arranged in levels, recent research has demonstrated that individual students are assessed regularly, and checklisted records of achievement against the achievement objectives are kept by most teachers. New assessment tools, such as exemplars of writing at each level and new standardised tests of reading and writing (for example, asTTle, ARBs and NEMP tasks) are being used by teachers to demonstrate that students in their schools are meeting the standards set out in the curriculum.
Such assessment practices have impacted on teaching in general. From the mid-1990s there has been a frenzy of assessment activity in primary classrooms, and teachers have moved from theme-centred teaching to genre-centred teaching to ensure that they cover the objectives in the curriculum. Thus, in most schools, long-term and unit planning reveal blocks of time devoted to "transactional" or "poetic" writing and "close" or "shared" reading.
In 2001 the Education Standards Act introduced the requirement that every school's charter will include a strategic planning section describing long-term goals for student achievement, an annually updated section setting out the school's specific targets for improved achievement. Schools will be required to discuss in their annual reports to their school communities any differences between the outcomes they have achieved and the targets that they earlier set themselves. From 2003, schools will be required to lodge copies of the annually updated part of their charter and their annual reports with the Ministry of Education.
While the intent of these changes is positive in that it leaves the nature of the standards to schools (rather than to national standardised tests that have proved less than successful in raising standards in other parts of the world), there is absolutely no acknowledgement that without better teaching and learning, teachers and parents cannot expect improved results. It's the old adage that no matter how many times you weigh the pig, it won't grow unless you feed it. We now have very substantial research evidence (for example, Black and Wiliam, 1998, "Assessment and classroom learning, Assessment in Education, 5, , 7-74) that shows that all the standards assessment, benchmarking or target-setting and checking in the world will not improve learning. Neither will a strategic, data-driven approach to school development if it is not linked to the improvement of formative assessment and teaching strategies in the classroom.
The use of AsTTle (version 4), currently being aggressively promoted in New Zealand primary and secondary schools, seems largely inconsistent with the practical implementation of rich formative interactions outlined above. This is unsurprising, given that AsTTLe is inherently contradictory because it tries to combine student-centered analysis with national norms. We would identify the following issues:
Do AsTTle tests assist in planning a teaching and learning programme based on authentic programmatic assessments that reflect the needs of individual learners and the community and institutional culture rather than from what children might be expected to be able to do at a particular age or year level as dictated by a standards-based, curriculum-linked prescription?
There is a question as to whether AsTTle does actually involve children in evaluating their own progress towards the intended learning? For example, does AsTTle promote self or peer assessment to increase the amount of feedback about, for example, how accurately pupils read something or how well they have conveyed a message in writing, or is time that might be devoted to this rich, formative assessment task spent instead in completing paper-based tests?
Does AsTTle remove opportunities for the meaningful authenticity of a writing conference that provide children with accurate and contextualized feedback about their work?
In short, is there a danger that AsTTle might focus teachers on standards rather than students, and may even detract from the way teachers listen to children and involve them in questioning processes.
AsTTle-generated tests of 10 or more pages administered on regular occasions may end up lowering children's self esteem and sense of achievement, not to mention teacher morale.
There must also be reservations about AsTTle, because the design of the assessment package produces items that lack ecological validity - there is no sense that the tests generated by the AsTTle package reflect the way effective teachers teach their topics. This dimension is fundamental to the establishment of ecological validity. Indeed, if the AsTTLe test ends up dictating classroom practice, the ecological validity of the tests may improve, but at the expense of teachers reflecting the teaching and learning strategies used to co-construct meaning with their students.Consequently, AsTTLe locks students into a culture of assessment-linked dependence rather than liberating them to use metacognitive strategies so central to the independent learner.
AsTTle is further problematic at a number of levels. Tests can dictate practice, but when the test is compromised, so too the practice compromised. And AsTTLe is compromised. Firstly, its construct validity is compromised because the construct on which it is based, the English curriculum, is fundamentally flawed. Secondly, its construct validity is compromised because the explanatory basis of documents, schema-centered cognitive theory, that supports the English curriculum is flawed. Thirdly, AsTTle is compromised because the learning theory implicitly adopted by the constructors of the test is inconsistent with effective teaching practice. Echoing the curriculum documents and NCEA is a notion that knowledge is best constructed and tested in particulate form and linked to standard-based assessment. This notion is the subject of on-going debate. AsTTLe is just one more example of knowledge becoming atomised into meaningless particles that prevent students from appreciating the whole. Reminiscent of the skills approach in the USA during the 1950s, and the partitioning in the New Zealand English curriculum, the test creates no-fly zones between and within subjects.
Fourthly, this separation and atomisation is further exacerbated by the analysis of scores provided by AsTTle, that includes comparison on several dimensions with national norms. In writing, for example, these are presumably linked to exemplars described in the Appendices of the test booklet. This analysis is problematic because of the potential for statistical misinterpretation - just what is a significant deviation from the norm and how is that assessed from box-plots by teachers with limited psychometric understanding? While the tests are reported to be reliable in a statistical sense, both the interpretation of exemplars and the interpretation and use of the test may be less so.
Further, is the interpretation of the contentious writing exemplars, based as they are on a functional systemic view of the writing process, an adequate or even warranted basis for the assessment of writing ability? Again this returns to the theme of this argument - does AsTTLe provide a valid measure of literacy in educational contexts?
Despite the comments above about recent policy moves, there is good news for improving literacy through assessment. Alongside the worldwide obsession with standards, accountability and centralised control, there has been an upsurge of interest in research into how assessment can be used to improve achievement rather than just measure it. In particular, formative assessment (or assessment for learning, interactive assessment, etc) has been shown to be particularly effective in raising achievement and improving learning (cf, Black and Wiliam, 1998; et al).
At the schoolwide level, implementing a strongly formative approach to assessment involves keeping standardised testing and assessment for accountability to a minimum and using rich tasks that encourage the full range of literacy and deeper learning rather than just testing skills and surface features. To support this, teacher appraisal and performance management systems that encourage the use of formative over summative uses of assessment and continued professional development should be implemented. Teachers need to be wary of software systems that require large amounts of test and levels-based information to be entered as these tend to drive teaching towards ticking and away from formative interactions. Such policies and practices can encourage classroom practices that support learning. Further details can be found in Hill's (2000) article in SET on this subject.
Improved learning and raised standards occur inside the classroom through rich formative interactions between teachers and students. In particular, literacy can be improved through:
- planning teaching and learning programmes from actual assessment results rather than what children might be expected to be able to do at a particular age or year level;
- sharing the learning intentions with the learners so that they know what they are trying to achieve and why;
- involving the children in evaluating their own progress towards the intended learning. Using self or peer assessment to increase the amount of feedback about, for example, how accurately they read something or how well they have conveyed a message in writing;
- giving children specific accurate feedback, both orally and in writing while marking their work.;
- involving children in target setting by informing them about what it means to be successful in, for example, writing stories that have a logical sequence;
- improving the way that teachers listen to children and the questions that they, as teachers, ask;
- monitoring children's achievement through generally agreed progress maps; and
- raising children's self esteem and sense of achievement.
For further information about these points, Unlocking Formative Assessment by Shirley Clarke is an excellent source.