Factsheet # 10: How does NZQA make sure internal assessment is fair and consistent across the country?

Download this Factsheet (PDF, 71KB)

Introduction

New Zealand is an optimum size to achieve nationally consistent internal assessment. We are large enough to have a complete system and a very sound pool of expertise – but we are small enough for all teachers in a subject to be in touch with the national system. We have a professional community of understanding about NCEA standards and processes.

Compared with teachers in many other countries, New Zealand teachers are assessment experts. A large proportion of teachers have had experience in setting and marking examinations, in writing national standards, and as moderators or as members of moderation clusters.

An important part of the quality assurance system for NCEA is called external moderation – making sure teachers are making consistent internal assessment decisions across the country. The aim is to equip teachers to make accurate and consistent judgements, by providing feedback and professional development.

How are teachers' assessment decisions checked for national consistency?

NZQA uses a number of methods to monitor the consistency and accuracy of internal assessment:

  • NZQA employs over 34 full-time equivalent moderators and 235 part-time moderators. Most moderators are current or recent teachers and all are assessment experts in particular subjects.
  • Moderators run best practice workshops, develop resources to guide schools and speak to meetings of subject associations. Moderators also check each school’s assessment tasks and activities, and the judgements schools are making when they assess student work. 
Moderators currently check a sample of about 10% of each school’s internally assessed work.This is ample to show whether or not teacher judgements are consistent across the country.
    •  
      • NZQA calculates agreement rates - measures of the extent to which moderators and teachers agree on whether samples of student work meet the standards (explained in more detail below).Each school receives a report on the quality of its internal assessment. These are called Managing National Assessment reports and they are published on the NZQA website. Moderation reports are quite specific. For example, for a particular achievement standard a school could be making entirely accurate decisions about Achievement and Merit, but getting it wrong in awarding Excellence.
      • NZQA compares each school’s results from internally-assessed and externally-assessed standards in each subject. Internal and external achievement rates differ nationally and it is expected that each school will broadly reflect national patterns. If a school’s internal results are greatly different from what is expected on the basis of their external results (and if the teacher-moderator agreement rates are poor) NZQA works with the school to improve its internal assessment processes.
      • As an ultimate sanction, NZQA can withdraw a school’s right to assess for national qualifications; this is most likely to apply to particular subjects within a school. In that case, provisions would be made for students to be assessed through another school’s quality systems.
      • Does the work meet the level described for Achievement? Should the student gain credits for that standard?
      • Has the student done well enough to get a grade beyond Achievement? If so, will it be Merit or Excellence? NZQA calculates two agreement rates - how well moderators and teachers agree on awarding credit for the standard, and how well they agree on the specific grade for that standard.
      • Agreement rates for awarding credit are always higher than agreement rates for grades, as statistically, fewer decisions are involved in deciding on credit.
      • In many standards there's a fine distinction between the grades of Achievement and Merit, and between Merit and Excellence. In 2010, 97% of assessment materials were deemed to be suitable, either unmodified or with only minor modification.This high figure reflects the fact that most schools now use downloaded assessment materials that have been pre-approved.

Moderation is a professional interaction

Moderators run assessment workshops for teachers in individual subjects, all around the country. In 2010, 173 workshops were held with 2180 attendees. In 2011, moderators will run assessment workshops for teachers in 27 subjects at 28 centres across the country.

Moderators work hard to provide teachers with "clarification statements" where there appears to be some difficulty or confusion about some aspect of the required standard.

Schools can ask for clarification or appeal a moderator’s decision. In 2009 and 2010, fewer than one in 1,000 moderator judgements were successfully appealed.

In many regions, schools voluntarily form clusters to enable teachers to compare notes with others teaching their subject.This is especially valuable where there are only one or two teachers of a particular subject in a school. In addition to enhancing NCEA assessment, clusters provide professional development for teachers.

Agreement rates

For each achievement standard, students receive a set number of credits if their work reaches the Achievement level. If they exceed the Achievement level, they can gain Merit or Excellence grades. So in effect, teachers make two decisions when they assess student work:

Across all standards at all levels, moderators agreed with 91% of teachers’ assessment judgements in awarding credit.This was up from 83% in 2009, and was higher than in any previous year.

2010 moderator/teacher agreement rates across all subjects:

Level 1
Level 2
Level 3
Across all Levels
Agreement on credit 94% 93% 91% 91%
Agreement on grades 86% 85% 82% 84%

Agreement rates for individual subjects can be found in the Annual Report on NCEA and New Zealand Scholarship Data and Statistics (2010) (PDF, 3MB).

Comments from the sector

"All teachers are part of the moderation system. It’s a process of being explicit about what we expect from students. It helps us – teachers, students, parents, the whole community – to better define what we want students to learn in our schools."

Professor Jeff Smith, University of Otago. Formerly Associate Dean of the Graduate School of Education, Rutgers University, New Jersey, USA

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz