Evaluation of NCEA Online

We’ve been giving students a chance to do some of their NCEA exams online since 2015 (through trials, pilots, practice exams, and end-of-year exams) and thousands have taken part. Students tell us that they prefer it. 

Digital learning and assessment are now well established with increasing numbers of schools and students taking part. Find out what they have to say.

2020 NCEA Online evaluation

NZQA’s findings from the 2020 NCEA Online evaluation show that the programme is well supported by schools and students, who continue to strongly enjoy the digital exam experience.

Students were asked to participate in an online survey after each digital exam to measure their self-reported satisfaction and experience. This feedback has been valuable and is already informing the next steps NZQA is taking with digital assessment, in collaboration with schools and the wider education sector.

Psychometric and statistical analysis of results, which was conducted Levels 1, 2 and 3 in English,  Level 1 in History and Level 2 in Business Studies, showed no evidence of disadvantage for those students sitting a digital examination, compared to the paper equivalent.

NCEA Online 2020 Summary Dashboard (PDF, 12MB)

Student Experience Report (PDF, 676KB)

  

2020 Psychometric Analysis Report

Statistical analyses compared 2020 student performance on the digital and the paper examination formats.

There was no conclusive evidence of a difference in achievement between these two groups of students that could be attributed to the examination format (digital or paper-based).

These findings are consistent with previous years, but for a larger group of students.  Examinations covered by the analyses in 2020 included English Levels 1, 2 and 3, History Level 1 and Business Studies Level 2.

The same analyses were conducted for Māori students who participated in the Level 1 English digital examination.  Their achievement was compared to a ‘matched ability’ sample of Māori students who did the paper-based Level 1 English examination.  There was no evidence of a difference in achievement between these two groups of students either.

For further information please email PRS@nzqa.govt.nz

Matched data analysis methodology detail

The matched data analyses are described in detail below.

A. Identify matching sets of students 

  1. Identify all digital-format students with at least one paper-format student with identical results in internal achievement standards at the same level.
  2. Identify all paper-format students that are eligible for matching to students identified in Step 1 i.e. paper-format students with at least one digital-format student with identical results in internal achievement standards at the same level.

 B. Generate 100 resamples and perform analysis

3. For each digital-format student identified in Step 1:

a. Identify all paper-format students with identical results in internal achievement standards at the same level.

b. From the list of paper-format students identified in Step 3a, randomly sample one student.

4. Calculate the grade distribution for externally assessed standards of matched paper-format students from Step 3.

5. Conduct Rasch analysis to estimate the difficulty parameters for externally assessed standards of matched paper-format students from Step 3.

6. Perform Steps 3 to 5 100 times.

C. Compile summary of results

7. Calculate the average of the 100 percentages, for each external standard-grade combination, generated from Step 6.

8. Calculate the mode (most commonly occurring value) of the 100 difficulty parameter estimates, for each external standard-difficulty combination, generated from Step 6.

9. Compare the results of Steps 7 and 8 with the corresponding values from matched digital-format students.

2019 NCEA Online evaluation

NZQA’s findings from the 2019 NCEA Online evaluation show that the programme is making good progress and students continue to strongly enjoy the digital exam experience.

Students were asked to participate in an online survey after each digital exam to measure their self-reported satisfaction and experience. This feedback has been valuable and is already informing the next steps NZQA is taking with digital assessment, in collaboration with schools and the wider education sector.

Psychometric and statistical analysis of results, which was conducted Levels 1, 2 and 3 in English and Level 1 in History, showed no evidence of disadvantage for those students sitting a digital examination, compared to the paper equivalent.

Trials and pilots 2015-2018

NZQA completed a range of trials and pilots between 2015 and 2018 to test our processes, build on our learnings and enable schools to evaluate their readiness to manage digital assessments. The findings of each of these are included in the following:

 

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz