Secondary assessment: getting the detail right
By Karen Poutasi, CE, New Zealand Qualifications Authority
27 September 2006 |
|
Post Primary Teachers Association annual Conference, Wellington
Tena koutou, tena koutou, tena koutou katoa
Thank you for that welcome.
As NZQA's relatively new Chief Executive, this is the first annual conference I've had the honour to attend.
The New Zealand Qualifications Authority takes very seriously its relationships within the education sector. Within that, secondary school teachers matter profoundly. You and your members are at the sharp end of education.
Those among you who teach the German language and German history will know the word schwerpunkt. In military matters, schwerpunkt means the focal point or the point of maximum effort in a battle, or more generally means at the crux of the matter. Your members have been and are at the schwerpunkt of educational innovation in New Zealand.
The title of this address is Secondary assessment: getting the detail right , and from what I've heard this morning, that is a mutual aim.
New Zealand's secondary teachers can stand very proud in terms of what they have achieved taking education forward in New Zealand. I don't say that tritely, but specifically and deliberately, in terms of the introduction into our schools of standards-based assessment, built around the National Qualifications Framework and the three National Certificates of Educational Achievement.
The broad goal has been, and still is, to have an education system that truly equips New Zealanders to meet the challenges and embrace the opportunities that the 21 st century global on-line economy offers. That is what you, the teachers, are creating.
The ultimate task of education is to grow human capital and capacity, and that takes place in that magic space between a teacher and a student. When I think back to my own schooling I can name two teachers who influenced my life, in one case profoundly. I had a French teacher who was fantastic. He could relate to kids. He loved French pop music and on Fridays we would sing the favourite French tunes on the hit parade! It's a great way to learn a language.
And I had a physics teacher who told me that the reason I couldn't do physics was because I believed I couldn't do physics! Despite my protestations, he was probably right. Under his guidance, I did well enough to enable me to pass stage one physics at university and of course that was mandatory to get into Medical School. So my debt to him is vast. Every person in this room has stories like that. That's real education - and it will always be so.
Indeed, as a medical doctor, I say the most basic injunction that an assessment system should embrace, like a doctor, is "First, do no harm". So, we have to make sure that the assessment systems that we run don't unduly interfere in what you do - "that magic space between a teacher and a student".
In net terms, what you've pioneered, in partnership with the rest of education sector, is an assessment system that is inclusive and places student learning at the centre. It also broadens the pathways that students can take to maximise their achievements. It allows schools to develop programmes and assessments that suit their students. It also allows for various modes of assessment to "suit the purpose". In other words, this is a flexible qualification system that suits our needs in the 21 century. It also means that fewer students now leave school saying "I'm dumb" or "I can't do anything".
However, there was always going to be criticism of NCEA. It was inevitable that many people would keep referring to templates from an earlier time. As has been said elsewhere "When faced with a totally new situation, we tend always to attach ourselves to the objects, to the flavour, of the most recent past. We look at the present through a rear-view mirror. We march backwards into the future." (Note 1)
It has also been hard, in that when the Scholarship problems emerged in early 2005, it was teachers in the senior schools who had both to explain and to plan for the following year, at least initially, in a state of uncertainty. The fact that teachers stayed on task is a testament to your dedication, professionalism and character.
However, touch wood, the tendency to say "In my day things were better" is fading. Standards based assessment, the NQF and the revised form of New Zealand Scholarship are settling - are becoming part of the education landscape that people work with and not against. If so, teachers deserve much of the credit.
What we can say with certainty is that each year the system gets better. That's because any assessment system depends on a myriad of judgements - judgements by those who write curricula, those who write standards, those who teach, those who set exams, those who sit the exams and those who mark the exams. This can never be formulaic. Early in an assessment system's life the unpredictability of those judgements make it highly likely that unintended consequences will occur when further changes are made. However, as time passes, we know better how everyone behaves within this new system and so the judgements and decisions we all make - teachers included - become better informed and as a result better in substance.
Last year we put in place some specific elements. Recall that we introduced a more sophisticated system for monitoring results during the marking process by comparing the results patterns with profiles of expected performance. This involved making a judgement prior to the exams being sat in terms of expected results.
This was not prescriptive, or normative, or backdoor scaling. We merely predicted that these patterns of results were likely, based on the standards themselves, the professional judgement of examiners and the historical evidence. Then, when marking began, we checked closely the initial pattern of results. If the results suggested something significantly different from the expected results then the marking was stopped and the reason found.
If that reason was to do with the performance of the students, then the marking continued without any alterations to the process.
If, however, the deviation from the expected performance was due to some rogue effect of the assessment instrument or of the marking system itself, then we acted. The marking was stopped, the marking schedule corrected, and then once we had checked everything the scripts already marked were remarked and marking from then on reflected the new judgements made. Last year, this procedure resulted in 17 - that is 5 per cent - of the 335 standards having some papers or questions remarked. We will use the same procedure this year.
Throughout all this, in acknowleding the contribution that teachers have made, I especially need to acknowledge the work that teachers do in internal assessment. In that regard, we rely on the continuing professionalism of teachers.
In fact, we know from the research that assessment is best done over time by those who know individual students face-to-face. There's two reasons for this. Internal assessment takes place throughout the year, whereas an end-of-year exam is a single snap shot, and we know that a student's capacity to perform ossilates. And, secondly, teachers know their students - from that magic space that I described - and can exercise judgement in a way that a marker examining a script cannot.
Indeed, I note the salient points that your president, Debbie Te Whaiti, makes in the September 2006 PPTA News: that despite the rhetoric we've had internal assessment for years. For example, Sixth Form Certificate was entirely internally assessed, with no moderation and no national standards as to what had to be studied. Debbie also notes that the marking of end-of-year examinations involves subjective judgements just as much as internal assessment does. In fact, experience shows the more valid the assessment, the more subjective the judgements in that assessment are likely to be.
Looking ahead, we are excited about changes that we propose to make to the Record of Learning that students get in their final year or on request. For some time we've been concerned that we aren't meeting the needs of employers who find our current Record of Learning confusing. Work is underway, including an on-line consultation. The main change being proposed is, broadly, to group standards into subject areas and levels. The current Record of Learning organises standards according to domains, which are counter-intuitive for most people. We want a reader to be able to quickly see the subjects the learner is strong in.
The other proposed innovation is that we put the learner's qualifications at the front of the document. The current Record of Learning has these down the back. Note that this is 'Qualifications and Other Achievements'. The current Record of Learning doesn't include University Entrance because technically it isn't a qualification. However, we've concluded that we should just tell people what they want to know. Feedback to date suggests we are on the right track.
We intend that an amended Record of Learning, once finalised, will be more useful, especially when presented by job seekers to employers.
We are likewise looking at updating the annual NQF Result Notice along the same lines. In essence the NQF Result Notice, if we go ahead, will look like a subset of the revised Record of Learning. The illustration laying out the proportion of candidates for each standard who achieved at each level and grade averages would be removed from the hard copy. Instead, in effect, the students would receive a Record of Learning extract covering the last 12 months.
We are considering this approach because we want to acculturate young people into life-long learning, as exemplified by the Record of Learning, and that will be something they will use for their whole adult lives. The vision is seamless life long learning and so we've resolved that right from the beginning the documentation annotating achievement should be framed in that way, for students to get used to.
I'd like to acknowledge the particular contribution that the PPTA is making to the debate about assessment for qualifications right now.
You have put a suite of interesting ideas to the Leaders Forum.
These include whether some new form of Leaving Certificate for students at the end of their schooling is required.
You've also asked whether finer graduations of assessment are required apart from just Not Achieved, Achieved and Achieved with merit or excellence and whether just having three grades is de-motivating.
You've raised the very big question of whether we really need to assess students for three years in a row.
You've asked whether there should be some way of assessing a student's generic capacity to integrate knowledge - a sort of "Can you see the big picture?" measure.
You've asked whether the assumption that 24 credits equals a year's study in a subject is now out of date, given that mixing and matching standards to create markedly new tuition programmes is growing apace.
You've asked whether some standards at level 1 need a new and separate status, in effect covering foundation topics.
These are all very good questions. I can't give you answers now - quite simply, we don't have them yet. Indeed, these are matters that involve the sector as a whole and, in some cases, government policy. What I can say is that your ideas are on the table. At the Leaders Forum and within the Government's education agencies, you are helping define the issues. Please keep doing this.
You are looking for robust research and extensive consultation and that fits with a comment I would make - that this is not a time for change for change's sake.
There is a demand in the sector for stability and for change only when it is properly planned and tested. So please don't expect wild swings in any direction. I don't think anyone is in the mood for aggressive adventures.
But do expect continuous, planned and deliberate improvements, in consultation with the sector. Let me emphasise, carefully planned and tested. As per the title of this address, with your help we are getting the detail right.
The Qualifications Authority is committed to being open and responsive. We have also better focussed our organisation to two externally focussed divisions - one spanning Qualifications and the other Quality Assurance. As you know, Bali Haque heads the new Qualifications Division. We are working on enhancing our communications with the sector and are interested in ideas on how we might do this.
The assessment systems used in secondary schools are not a finished product by any means and they are not perfect. The NQF will never stop growing and changing - in a sense it is organic. Likewise, NCEA and Scholarship will always be a "work in progress". That's what continuous improvement means.
However, it seems to me that we have reached a "tipping point". A consensus has developed around the substantial issues; there's acceptance that our current arrangements are here to stay and that we have achieved a vast improvement on what came before. Our arrangements are credible overseas and, best of all, they are liberating for schools. More students are achieving success as a result, and that's the bottom line.
Thank you.
Marshall McLuhan, The Medium is the Massage, 1967
Page updated: 28 September 2006