2007 NCEA Survey Results
Survey results of teachers’ and candidates’ views of the 2007 NCEA and Scholarship assessments, and of Principals’ Nominees views on the procedures surrounding the NCEA.
The following are summaries of the results of four surveys that the New Zealand Qualifications Authority (NZQA) undertakes annually following the completion of National Certificate of Educational Achievement (NCEA) and New Zealand Scholarship exams for that year. The full results can be obtained from NZQA by contacting:
Research and Knowledge Services
New Zealand Qualifications Authority
monitoring-research@nzqa.govt.nz
Please note that this summary report is not intended to provide an in-depth analysis of differences and trends identified; rather it records these matters, so that emerging issues and situations can be further investigated in extension reports.
Interpretations of the findings should consider the level of response to the surveys, especially of the smaller response rates, as these may result in potential bias.
2007 Survey of NCEA Candidates - Summary
In 2007, candidates being assessed for National Certificate of Educational Achievement (NCEA) level 1, 2 and 3 standards were given the opportunity to comment on their assessment experience. The survey was posted to the home addresses of 1,800 randomly selected candidates entered for at least one externally assessed standard at level 1, 2 or 3 (600 students were selected at each level). A total of 248 surveys were returned, representing a response rate of 14%.
The 2007 survey sought information on candidates' profiles, experiences of external assessments, perceptions of internal versus external assessment, perceptions of external assessment supervision and perceptions of endorsed certificates.NCEA respondents' profile
- More females responded to the survey than males, particularly at level 3 where 79% of those who returned surveys were female.
- Across levels 1, 2 and 3, the majority of respondents (84%) entered for 80 credits or more with most candidates (58%) entering between 80 and 119 credits.
- Overall, 69% of respondents entered for over 20 unit standards. On average, level 1 respondents attempted a higher number of unit standards than level 2 and 3 respondents (39, compared with 31 and 29).
- While only 4% of level 1 respondents had withdrawn from standards once they were entered for them, this increased to 26% for level 2 and 50% for level 3 respondents. The most common reasons given for withdrawing were that they found the standard too difficult, they did not have the time to understand the standard, or that they did not think they would perform well; therefore they sat the standards they were more likely to achieve.
External assessment experience
- Overall, 64% of respondents reported they felt well prepared for 'all' or 'most' external assessments.
- Overall, 39% of candidates ran out of time in an examination session. This varied greatly by the level studied, with 67% of those studying at Level 2 and 3 running out of time compared to only 22% of Level 1 candidates.
Internal versus external assessments
- Overall, 50% of respondents preferred internal assessment to external assessment. The proportion was slightly higher among level 3 respondents (61%) and slightly lower among level 1 respondents (38%). Reasons for preferring internal assessment included that it was less stressful, that they had more time to prepare, that the subject was fresh in their minds when assessments were undertaken, and internal assessments were spread throughout the year rather than in one concentrated examination period.
- Respondents were fairly evenly split over whether they believed internal or external assessment was a better measure of what they had learnt during the year, with the exception of level 2 respondents who considered internal assessment or an even mixture to be a better measure of learning. Forty-one percent (41%) of respondents did not think it was important whether their credits came from internal or external assessment.
External assessment supervision
- Respondents were positive about the examination supervision, reporting that the supervisors gave clear instructions before each exam started and that the supervision was quiet and non-distracting.
NCEA endorsed certificates
- Overall, 68% of respondents said the possibility of gaining a merit or excellence endorsement on their NCEA certificate gave them at least a small increase in motivation to achieve higher grades. The most common reasons respondents gave for being motivated by NCEA certificate endorsements were that they felt a greater sense of personal achievement/motivation, they would receive recognition that they had worked harder or achieved higher, or that it would assist them in the future (e.g. for getting into university or including in their CV).
2007 Survey of SCHOLARSHIP Candidates - Summary
In 2007, candidates sitting Scholarship examinations were given the opportunity to comment on their assessment experience. The Scholarship candidates' survey was posted to the home addresses of 500 randomly selected candidates entered for at least one Scholarship subject. A total of 101 surveys were returned, representing a response rate of 20%.
The 2007 survey sought information on candidates' profiles, Scholarship examination experience and examination supervision.Scholarship respondents' profile
- Most Scholarship respondents were Year 13 students (92%); the remainder were in Year 12. More females (62%) responded to the Scholarship survey than males (38%).
- Candidates most commonly sat one Scholarship examination (45%), although 14% sat Scholarship in four or five subjects. Candidates commonly reported that they chose to sit Scholarship for the monetary awards, personal challenge, in order to prepare themselves for University and because of teacher encouragement.
- Scholarship candidates on average attempted 119 credits, slightly higher than the average for Level 3 candidates (102).
- One-in-four candidates withdrew from a Scholarship subject after they had entered it. The main reason candidates gave for withdrawing was that they lacked the time to study for the exam(s) and that they found the subject too difficult.
Scholarship examination experience
- Approximately half of Scholarship respondents considered the level of difficulty of Scholarship examinations to be "about right", but 31% rated them as "difficult" or "very difficult"
- Two-fifths reported they were "well" or "very well prepared" for their assessments. A further 43% reported they were "kind of prepared".
- Sixty seven percent (67%) of respondents reported that the time allocated for the exams was "about right", while 32% of respondents considered the time for exam "very short" or "short".
- Twenty six percent (26%) of respondents left at least one Scholarship examination early.
- Respondents' suggestions for improvements to Scholarship examinations included the timing of the Scholarship examinations within the overall examination schedule, with the majority of respondents suggesting that Scholarship should take place after “normal” exams
Scholarship examination supervision
- Scholarship respondents rated the supervision highly, with 92% agreeing that clear instructions were given before each exam started, and 100% agreeing that supervision was non-distracting
2007 Teachers' survey - Summary
In 2007, teachers were given the opportunity to comment on the NCEA and Scholarship external assessments. Five copies of the level 1, 2 and 3 teachers' surveys were sent to 430 secondary schools inviting teachers to comment on their subject's examination papers. Schools with students entered for Scholarship (295) were also sent five copies of the Scholarship questionnaire. Comments were requested for all subjects at level 3 and Scholarship, while selected subjects were chosen at levels 1 and 2.
This year for the first time teachers were able to identify if they gave group or departmental responses. Group responses for level 1, 2 and 3 NCEA teachers' surveys were on average 38%, 38% and 36% respectively while 42% of the Scholarship teachers gave group responses.
A total of 1558 surveys were returned from NCEA teachers and 355 surveys were returned from Scholarship teachers. Individual subject reports have been produced at each level and provide specific detail on the responses to each standard.NCEA Teachers' Survey
- Overall feedback from teachers was positive about the 2007 NCEA external assessments. Teachers generally agreed that the tasks provided good coverage of the achievement criteria and that questions were communicated clearly and precisely.
- Most agreed that the instructions were clear and appropriate, and that the assessment specifications were helpful.
- The answer books were rated highly by teachers in terms of planning space, space for full written answers, and room for drawing diagrams. Similarly, resource materials (diagrams, photos, etc., and accompanying information) were also rated highly.
- Averaging across all standards and levels, the majority of teachers believed the suggested time to complete all questions in the assessment was “sufficient”. Of the remaining teachers, more likely to report that the suggested time was “too short” rather than “too long”.
- The reading level was generally rated as “suitable” for the year level of students, with slightly fewer teachers rating the overall difficulty level as “suitable”.
- Few teachers reported timetable clashes with their subject. Where clashes did occur, the schools managed them by holding one of the examinations in the morning, the other in the afternoon and providing supervision during the break. With students starting their exams early, they could fit three exams in one day, particularly having the NCEA coordinator take care of it.
Scholarship Teachers' Survey
- Approximately nine-in-ten teachers agreed that there was appropriate coverage of the subject, and that questions depth and breadth were appropriate. There was also strong agreement that questions were communicated clearly and precisely.
- While most agreed that the instructions were clear and appropriate, fewer agreed that assessment specifications and assessment circulars relating to the format of the assessments were helpful.
- The answer books and resource material were rated highly by teachers in terms of planning space, space for full written answers, room for drawing diagrams, and clarity of resources (maps, photos etc.).
- Averaging across all subjects, around three-quarters of teachers believed the suggested time to complete all questions in the assessment was “sufficient”. The majority of remaining teachers reported that the suggested time was “ short ” or “t oo short ”.
- Around three-quarters rated the overall difficulty level as “suitable” for Scholarship students, with 23% of teachers stating the examinations were “a little too difficult” or “very difficult”.
2007 Principal's Nominees End of Year Survey - Summary
In November 2007, an email was sent and a paper survey distributed via circular (SecQual Circular S2007-038) to all 441 Principal's Nominees inviting them to give feedback on the various procedures surrounding the NCEA. A total of 310 surveys were returned, representing a response rate of 67%.
The survey consisted of three parts: school liaison, monitoring and reporting, external moderation, and the process for entries and results.School Liaison, Monitoring and Reporting
-
Eighty seven percent (87%) of responding schools have participated in at least one visit/meeting with their School Relationship Manager (SRM) in 2007. The most common visit was a meeting with the Principal's Nominee (83%), followed by staff/HOD/department meetings (26%) and Managing National Assessment (MNA) visits (22%).
- Having the SRM as the first point of contact was highly regarded by Principal's Nominees, with the majority rating this as "useful" or "very useful" . Principal's Nominees were also positive about the various SRM visits (such as the MNA visits and staff meetings) with 98% rating them as "useful" or "very useful".
- Most information sources the survey asked about were rated as "useful" or "very useful" by at least 96% of Principal's Nominees; this included the SRM information emaiLink, the school login page, the NZQA website, and the SecQual and Assessment Matters circulars. Scholarship circulars were rated as 87% "useful" or "very useful", with some Principal's Nominees commenting that they did not have students entered for Scholarship in 2007.
- Eighty eight percent (88%) of Principal Nominees stated that they had attended one of the Principal's Nominees Seminars held in February/March 2007. Of these, 97% found the seminars "useful" or "very useful".
External Moderation
- The majority (92%) of Principal's Nominees rated the annual National Moderator Report for subject teachers as "useful" or "very useful".
- Most Principal's Nominees considered external moderation to be useful in providing reassurance for teachers and for informing future assessment practice.
- Over half of respondents (64%) reported that individual subject moderator reports were returned in a timely manner "most" or "all of the time".
- Two years ago, NZQA changed the format of the NCEA Assessment Reports (formerly the Examiner's Reports) to include more bulleted points and be more focused on identifying essential skills and knowledge required for the examinations. Fifty percent (50)% of respondents preferred the new format of the Assessment Reports.
Entries and Results
- Fifty five percent (55%) of schools submitted entries via the website and 45% submitted them electronically by email. Half of those submitting entries and results electronically used the MUSAC software.
- Most rated the process for submitting entries and results to NZQA as "good" or "excellent". The responsiveness of NZQA staff to queries about entries and results was rated highly, with 95% rating responsiveness as "good" or "excellent'.