National External Moderation Summary Report for 2008, Semester 2

Prescription: 630 Leadership

Introduction

This report provides a national perspective on the moderation of 630 Leadership.

Assessment materials from 16 Tertiary Education Organisations (TEOs) were moderated for this prescription.

Of the 16 submissions:

four submissions met the national standard
eight of the 12 submissions that did not meet the national standard did not completely assess all learning outcomes.

Other areas where the national standard was not met included the following, with the number of submissions concerned shown in parentheses:

learning outcomes not assessed at an appropriate level as specified in the prescription (five)
prescription weightings not adhered to (eight)
lack of clear and appropriate assessment conditions and instructions (seven)
assessments lacking consistency with current good industry practice (three)
marking schedules with insufficient detail to ensure consistent marking (three)
marking schedules inconsistent with the requirements of the prescription and assessment materials (six)
inconsistent marking of candidates' work according to the marking schedules (six).

Of the 12 submissions not meeting the national standard, five did not meet four or more different requirements.

Of the four submissions that met the national standard overall, none met all requirements.

Presentation of assessment materials

Most submissions were well presented, bound and clearly divided according to the checklist. Only two submissions were incomplete and this was because they did not include learner samples for substantial assessments.

Two submissions included internal moderation forms that are not required for national external moderation.

Assessment grids

All submissions included assessment grids. The more detail they provided, such as the specific parts of assessments that applied to particular key elements and subcomponents of learning outcomes, the more useful they were to the moderators. However, not all assessment materials accurately reflected the information in the assessment grids. This suggested that grids were not always being used as an assessment design tool, which may have contributed to some of the submissions reflecting incomplete assessment of learning outcomes.

Mark allocations specified in assessment grids should be rigorously adhered to so the allocations of marks to actual assessments match those stated in the assessment grid.

Learning Outcomes

Learning outcomes indicate assessment outcomes and specify what learners need to know and be able to do. Key elements indicate assessment coverage and specify how the related learning outcome should be evidenced. Key element assessment evidence should be provided in the context of the learning outcome.

Incomplete assessment of learning outcomes and their key elements and sub-components was the most common fault with submissions. In some cases, key elements and sub-components were not assessed in the context of the learning outcome.

It is implicit that every key element sub-component must be covered in assessment, and options are available only for learning outcome 3, key element a) ‘self-assessment process'; and learning outcome 4, key element b) ‘different business entities'. Some providers assessed both options in learning outcome 3 a), or did not require a suitable comparison of entities in learning outcome 4 b).

Level of assessments

Assignments, especially those encouraging engagement with real business leaders and opportunities for reflective thinking, worked well. Tests generally did not encourage higher-level thinking, but rather recall and comprehension. When answer sheets provided boxes or frames for answers, these further limited the learners' scope for achievement at level 6.

Learning outcome verbs indicate the level of activity required. For example, learning outcome 4 states, ‘Students will research, compare and evaluate how leadership concepts are applied'. However, some assessments did not provide opportunities for learners to "research", "compare" and "evaluate" the application of leadership. In some cases, learners were repeating theory that they had already engaged with for learning outcome 1.

Prescription weightings

A 10% aggregate variance is allowed in assessment weightings. That is, the percentage variation in total across all learning outcomes should not be more than 10%.

Seven submissions did not meet the weighting requirements; however, this issue alone did not result in a submission not meeting the standard. For example, more than one submission assessed learning outcome 1 a second time in an assessment that, according to the assessment grid, was for learning outcomes 3 or 4. Incomplete assessment of learning outcomes and key elements led to over-emphasis on the elements that were assessed. Some submissions allocated significant marks to items not directly related to the prescription, but which related to the level and the type of assessment. For example, they awarded marks for team work, presentation, structure, and referencing. When marks for these items reach more than 10% across all assessments, prescription weightings were significantly distorted.

Assessment conditions and instructions for learners

Moderators have no way of knowing what information is available to learners about assessments beyond what is provided in submissions. Lack of clarity and appropriateness in assessment conditions and instructions were an issue for seven submissions. The main problems were not providing learners with a clear idea of how to carry out the assessment to address the learning outcomes being assessed, not providing appropriate assessment conditions to meet the learning outcomes being assessed, and not providing clear criteria and mark breakdowns for learners to see how to gain available marks.

Some TEOs were providing useful activities for learners on reflective thinking, especially with journal writing (in relation to learning outcome 3). On the other hand, an examination with its time constraints does not provide optimal conditions for a learner to reflect on self-awareness and analysis of their leadership potential.

Consistency with industry practice

Moderators looked for assessments at an appropriate level, that reflected up-to-date concepts, and that assessed skills, knowledge and values appropriate to business/industry in New Zealand . For example, one submission had such a high proportion of group assessments that the importance of individual responsibility was lost. Good practice assessments included research, encouraged critical thinking and considered ethics. Only three submissions did not meet this requirement.

Marking schedules

Only three submissions did not provide sufficiently specific marking schedules to enable consistent marking. H alf (eight) provided marking schedules that were inconsistent with the requirements of the prescription and assessment materials.

In some cases, marks information provided to learners with assessments was different from the marking schedule used to mark the work. In two cases, this type of discrepancy related to examinations. As mentioned above, clear and specific marking schedules (without suggested solutions) provide useful guidance for learners as they prepare for their assessments.

For subtotals where marks are greater than five, it should be stated clearly in the marking schedule how such subtotals are to be broken down for consistent and fair marking. In other words, there has to be a rational way to allocate a range of marks for learners who fully complete as opposed to those who only partially complete their responses to questions/tasks. The assessment schedule should be sufficiently detailed and clear to allow an assessor other than the assessment designer to mark learners' work.

When specific components of the learning outcomes are not mentioned in the marking schedules, moderators have to assume that they are not being marked. One marking schedule for assessing learning outcome 3 mentioned the learning outcome only once, allocating an optional half a mark for it. This marking schedule had instead become focused on the process of journaling generally.

Learner work samples

The samples were checked to see that marking was consistent with marking schedules. Seven submissions did not meet this requirement. Where there were problems with marking schedules, it was often difficult to confirm that consistent and fair marking had taken place. Two submissions contained obvious examples of plagiarism that were apparently not identified by assessors.

Most assessors showed good practice in handing back assessments with detailed marking sheets and constructive comments.

Conclusions

The main issues were incomplete assessment of the learning outcomes, key elements and sub-components; marking schedules that were inconsistent with the requirements of the prescription and assessment materials; prescription weightings not being adhered to; unclear assessment conditions and instructions; and inconsistent marking of learners' work.

Page updated: 16 June 2009