Marker report

Evaluation Report 2018 (English, Classical Studies, Media Studies)

1. Introduction

The marker survey was designed to measure markers’ self-reported experience of marking a Digital Pilot examination, including their experience of the marking tool.

Of the 56 markers who marked digital examinations, 33 completed the online survey ‒ a survey response rate of 59%. The survey consisted of 13 questions, two of which were open-ended. The respondents did not all answer all the questions in the survey.

The markers who answered the survey had an average of 22 years of teaching and an average of 13 years marking for NZQA.

Below is a summary of the main findings. Appendix 1 is a full list of the survey questions. Appendix 2 is a count of responses for each closed question.

2. Survey findings

2.1 Overall satisfaction

As stated in section 1 above, this survey had a response rate of 59%. Overall, markers were positive about the digital marking process. 82% (27 of 33) of respondents either agreed or strongly agreed the digital marking process was satisfactory. This percentage was less than in 2017 when 94% (59 of 63) had the same sentiment.

2.2 The marking tool and support provided

91% (30 of 33) of respondents agreed or strongly agreed that they had received enough training to confidently mark online. This is an increase from 2017 and 2016 when 84% (52 of 62) and 71% (15 of 21) of respondents respectively agreed or strongly agreed that they had received enough training to confidently mark online.

85% (28 of 33) of respondents agreed or strongly agreed that the marking instructions for the digital examination were easy to follow. This percentage was less than in 2017 when 92% (59 of 64) had the same sentiment.

76% (25 of 33) of respondents agreed or strongly agreed that the digital marking application had all the tools they needed for marking. This percentage was less than in 2017 when 83% (53 of 64) had the same sentiment.

86% (25 of 33) of respondents agreed or strongly agreed that the digital marking interface was easy to use. This percentage was an increase compared with 2017 when 81% (52 of 64) had the same sentiment.

25% (8 of 32) of respondents experienced technical issues when marking the digital examination. This is a decrease from 2017 and 2016 when 57% (36 of 63) and 75% (15 of 20) of the respondents respectively experienced technical issues when marking the digital examination.

2.3 The online marking experience

87% (27 of 31) respondents agreed or strongly agreed that there was sufficient support from NZQA during digital marking. This is an increase from 2017 and 2016 when 84% (51 of 61) and 71% (15 of 21) of respondents respectively agreed or strongly agreed that there was sufficient support given during digital marking.

97% (31 of 32) of respondents agreed or strongly agreed that the assessment schedule was easy to follow online. This is a slight increase on 2017 and 2016 when 95% (59 of 62) and 89% (17 of 19) of respondents respectively agreed or strongly agreed that the assessment schedule was easy to follow in a digital environment.

82% (27 of 33) of respondents agreed or strongly agreed with the statement that their marking of digital assessments produced the same results as it would have for identical assessments completed on paper. This percentage was less than in 2017 when 98% (62 of 63) had the same sentiment.

33% (11 of 33) of respondents agreed or strongly agreed that the marking of digital examinations took less time than the paper examination. This was a decrease from 2017 and 2016 when 75% (47 of 63) and 62% (13 of 21) of the respondents respectively agreed or strongly agreed that the marking of digital examinations took less time than the paper examination.

2.4 Further suggestions/feedback

Respondents were asked if they had any further comments or suggestions about their experience marking the Digital Pilot examination. Most mentioned that digital essays tend to be much longer than essays on paper, which contributed to more time spent marking:

"Many students wrote a lot more than necessary. I needed to take more time to consider each response more carefully because it was easier to read so students seemed to achieve more readily."

"Students wrote significantly more in the digital essays than on paper. Very few of them planned their essays."

The longer essays contributed to eye fatigue and headaches amongst the survey respondents:

"I would have liked the ability to change the line spacing when marking and alter the font size. Being presented with a large block of text proved uncomfortable to mark after a period of time."

"It was tiring on the eyes and did get the odd headache and sore eyes from looking at the screen all the time. You couldn't enlarge the students' responses and often they were completed in a smaller font-10."

The survey respondents also commented and had feedback on the interface:

"Papers need to be allocated to the individual marker, rather than the markers scrolling through to find their candidate numbers. They should be grouped according to markers like the paper allocation. There should be a way to see progress of page so you can see how far through you are."

"It's a minor issue, but it would help if there were a search function so that we could get straight to our allocated pages, rather than have to use the 'page forward' icon every time we log in."

"Someone else marked some of my allocation so I thought I was finished but wasn't and had to marks some additional papers at the end. The sorting function clearly didn't block papers off for markers."

Generally positive comments included:

"Digital scripts easier to manage and read."

"… the digital marking process was straight forward."

"I appreciated the opportunity to mark digital responses, as it gave me valuable insight into how digital exams are constructed."

 

For more information, download the full report (PDF, 115KB).

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz