Terms of Reference

23 Dec 2016

Review of the causes of the error in the 2016 NCEA Level 3 Statistics examination (standard 91585) and of the quality assurance processes for this examination and the examinations for the Mathematics and Statistics standards 91037 (Level 1), 91262 and 91267 (Level 2), and 93201 (NZ Scholarship Statistics) 

Background

The New Zealand Qualifications Authority (NZQA) is responsible under the Education Act 1989 for managing the external assessment of secondary school students. The external assessment comprises the National Certificates of Educational Achievement (NCEA), Levels 1-3, and New Zealand Scholarship (Scholarship). Mathematics and Statistics is assessed at NCEA Levels 1 and 2; for NCEA Level 3 and Scholarship, Statistics and Calculus are assessed separately.

In 2016, Question 3(b) in Level 3 Statistics examination paper standard 91585 contained a critical, undetected error that made the initial question and the following parts of the question unworkable. Consequently, candidates were unable to answer the question.

Mathematics teachers have also expressed concerns about the adequacy of quality checks on examination papers for standards 91037 (Level 1), 91262 and 91267 (Level 2) and New Zealand Scholarship Statistics (93201), which did not detect discrepancies in, or relating to, a number of tables and graphs.

Given the significance of the error in the examination paper for standard 91585 and its impact on candidates, and the concerns relating to the quality checks on the four other Mathematics papers, NZQA has determined to convene an independent expert panel to review the respective causes. The Panel is asked to make appropriate recommendations to NZQA on how such situations can be prevented in future external examinations in Mathematics and Statistics. 

Objectives

The Review Panel has been established to:

  • consider the established procedure for developing external examination papers from initial contractor appointments, development of examination papers, through to approval to print and the current arrangements for quality assuring draft papers;
  • determine whether the examination papers for standard 91585, in particular, and standards 91037, 91262, 91267 and NZ Scholarship statistics 93201 complied with the established procedures, timelines and quality assurance arrangements;
  • determine, if they did not, why they did not follow the established procedures, and determine the causes of the problems and why they were not identified; and
  • recommend to NZQA, in light of the 2016 issues in Mathematics and Statistics, improvements to the procedures for the development of future external examination papers and their quality assurance.

Scope

The scope of the Review includes all stages in the system NZQA uses for the development of external examination papers from the appointment of contractors, the initial draft through to the approval to print and the roles played by the various Examination Critiquers. The Review Panel will ascertain through documentation and discussions with appropriate NZQA staff whether the examination papers for standards 91585, 91037, 91262, 91267, and NZ Scholarship Statistics 93201 went through all the established stages for examination paper development and quality assurance by the Critiquers. The Panel will determine if there were issues relating to development, timelines and quality assurance that led to the error and concerns with the quality checks in the other Mathematics examination papers. 

Having ascertained what factors led to the problems with these papers, the Panel will consider and make recommendations to NZQA on how such problems might be prevented in future years and whether changes or improvements in the current quality assurance processes are required.

The following are out of scope for this review:

  • Any matters relating to Standards other than the Mathematics Standards specified in the Scope of the Review[1]
  • Changes to the fundamentals of the external examination system, such as the NCEA and NZ Scholarship assessment structure, supervision, marking and results procedures, and certification.

Timeframes and Reporting

The Panel will meet in late January / early February 2017 and its final report should be submitted to NZQA by 28 February 2017.

The report will be published in early March 2017.

Panel Members

  1. Dr Sharleen Forbes (Convenor): Former Adjunct Professor of Official Statistics, School of Government, Victoria University and has held a number of senior research and policy positions within the public sector.  Dr Forbes is a current member of NZQA’s external Technical Overview Group Assessment (TOGA) which advises NZQA on the assessment process for NCEA and New Zealand Scholarship.
  2. Marion Steel (Panel member): Current Head of Department Statistics and acting Head of the Mathematics learning area at Epsom Girls Grammar School.  Ms Steel is a member of the New Zealand Association of Mathematics Teachers and the New Zealand Statistical Association.
  3. Linda Tame, QSM (Panel member): Former Principal of Lincoln High School and current Pro-Chancellor of the Lincoln University Council.

Support for Panel

An advisor and secretariat will be provided to the Panel.



[1] NCEA Level 3 Statistics Standard 91585; NCEA Level 1 Mathematics Standard 91037; NCEA Level 2 Mathematics Standards 91262 and 91267; and New Zealand Scholarship Statistics 93201

Find information for...

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz