Monitoring of New Zealand Diploma in Business (Level 5 and 6): 2018 summary

In 2018 NZQA began monitoring the programmes leading to New Zealand Diplomas in Business (Levels 5 and 6) [Refs: 2459 and 2460].

This summary (PDF, 216KB):

This summary of programme monitoring is mainly for:

  • tutors and assessors
  • programme developers and programme leaders
  • academic and quality managers.

There are also top tips for these three groups.

New Zealand Diplomas in Business (Level 5 and 6)

The New Zealand Diplomas in Business (Level 5 and 6) [Ref 2459 and 2460] were published on the New Zealand Qualifications Framework in 2015. This followed a mandatory review of Business qualifications from 2012-2014.

The New Zealand Diplomas in Business (Level 5 and 6) [Ref 2459 and 2460] replaced the National Diplomas in Business (Level 5 and 6) [Ref: 1498 and 1499]. The new and replaced Level 5 qualifications are focused on operational management, while the Level 6 qualifications are more strategic.

The National Diploma in Business (Level 5) was a standards-based qualification, intended for people who held, or aspired to hold, positions involving managing and leading a team or business unit.

The National Diploma in Business (Level 6) was also a standards-based qualification, intended for people who held, or aspired to hold, positions managing and leading several different(but allied) teams or business units. Some of the standards in each qualification meant the candidate needed to be assessed in their workplace.

As a result of the mandatory review a third business qualification, the New Zealand Diploma in Business (Level 6) [Ref: 9003], was also designated expiring.

The New Zealand Diploma in Business was a prescription-based qualification. It allowed learners to gain a broad range of general business skills and knowledge. Learners could then focus on career-related options such as accountancy, banking, management, marketing and tourism. This qualification was very popular with domestic and full-time international learners.

NZQA monitored the quality of assessment practice in programmes leading to all three of these Business qualifications through the annual process of national external moderation. The quality of assessment practice was variable. In some cases, NZQA withdrew accreditation from providers due to their ongoing non-compliance with national external moderation requirements.

The new Business qualifications are not solely academic or theory-based, but are firmly rooted in the practical requirements of New Zealand business and businesses. This means that programme design and delivery, and all assessment, must be conducted in and for the context of a real or realistic business entity, and must reflect the requirements of that context.

NZQA considers programmes leading to the new Level 5 and 6 Business programmes a priority for monitoring.

This is because there are:

  • a large number of providers offering the programme, with diverse delivery and assessment methods
  • a large number of domestic and international learners enrolled

concerns about programme quality identified from other NZQA quality assurance activities (including national external moderation of previous Business programmes, and programme monitoring of Level 7 Business Diplomas). 

Monitoring activities

In 2018, NZQA monitored:

  • 21 out of 42 approved Level 5 programmes leading to Level 5 New Zealand Diploma in Business [Ref: 2459], and
  • 11 out of 30 approved Level 6 programmes leading to Level 6 New Zealand Diploma in Business [Ref: 2460].

The 32 programmes NZQA monitored were delivered by 21 tertiary education organisations (TEOs).

For the 2018 monitoring, NZQA used the following three types of monitoring:

  • Moderation only: NZQA moderated a sample of assessed learner work.
  • Desk evaluation: NZQA evaluated the programme documentation that TEOs supplied, including NZQA moderation.
  • Visit: NZQA made a site visit, including interviews with relevant staff members and learners, as well as a desk evaluation.

The 32 programmes were monitored as below:

  Level 5 Level 6
Moderation only 6 1
Desk evaluation 2 1
Visit 13 9

The outcomes of NZQA’s monitoring activities

NZQA found that many of the programmes did not meet the criteria for programme approval and accreditation.

Of the 21 Level 5 programmes monitored:

  • two programmes met programme criteria overall
  • a further five programmes met some programme criteria with some areas requiring remedial actions to meet programme approval and accreditation criteria
  • the remaining 14 programmes did not meet programme criteria overall and required significant remedial action. In some cases, NZQA made a statutory intervention.

Of the 11 Level 6 programmes monitored:

  • no programme met programme criteria overall
  • a further three programmes met some programme criteria with some areas requiring remedial actions to meet programme approval and accreditation criteria
  • the remaining eight programmes did not meet programme criteria overall and required significant remedial action. In some cases, NZQA made a statutory intervention.

Common issues

The most common issues which prevented programmes meeting programme criteria included:

  • poor assessment and moderation practice
  • unapproved changes to the programmes
  • a lack of robust programme review.

The rest of this report explains the monitoring findings in more detail.

Findings

Monitoring process

TEOs delivering the Level 5 and 6 business programmes have cooperated with the monitoring process. Most of the providers gave NZQA the requested documentation, appropriately labeled, and access to relevant staff and learners. NZQA acknowledges and appreciates the time and effort it takes for TEOs to participate in programme monitoring.

Assessment and moderation

This section of the monitoring report addresses:

  • assessment methodology and procedures
  • authenticity
  • moderation systems and processes
  • the results of NZQA’s moderation of a sample of assessed learner work.

Assessment methodology and procedures

Most of the Level 5 and 6 business programmes that NZQA has monitored use an achievement/grade-based marking system. NZQA approves programmes with assessments that are competency/standards-based or achievement/grade-based.

Both the learner and the assessor must have clear guidance on the level of performance required to meet each learning outcome. Assessing against learning outcomes that map to the graduate profile is central to assuring the quality and integrity of programmes.

Expectations for achievement/grade-based assessment

For achievement/grade-based assessment, NZQA moderators expect to see detailed criteria for each grade. The criteria for the minimum passing grade (typically a ‘C-‘or 50 per cent) must align with achieving the learning outcome(s) within the assessment(s).

Grades higher than a ‘C’ show work over and above what is needed to achieve each learning outcome.

Guidance on assessments for multiple learning outcomes

If the assessment task covers multiple learning outcomes, the assessor must ensure that the learner evidence meets the minimum passing criteria for each learning outcome.

For instance, if an assessment covers three learning outcomes, a learner might do exceptionally well on two learning outcomes, achieve a mark of more than 50 per cent, but has not achieved the third learning outcome.

This is not appropriate, as learners should not pass if they have not provided sufficient evidence for each of the learning outcomes being assessed.

Assessment methods

In the Level 5 and 6 business programmes NZQA monitored, assessment often consisted of two to four assessment tools of varying weightings for each programme component/module.

Assessment methods varied across components between programmes. They included:

  • written reports
  • presentations
  • New Zealand-based case-study analyses
  • group-based assessments
  • examinations and portfolio submissions.

In some cases, the assessment method used was not relevant to the learning outcomes assessed: for example, using an exam with multiple-choice questions when the learning outcome required explanation and implementation of a certain concept.

Using relevant assessment tools is critical to designing assessments that ensure learners meet certain learning outcomes.

Group work and group assessment

Every programme monitored included group assessments. Group assessment is different to group work. In group work, learners collaborate as part of the learning process, but they are assessed individually. In group assessment, learners are typically awarded the same assessment result for something they produced collectively.

Group assessment is a valid method of assessment for Business programmes. It is an overarching condition of these qualifications that assessment must be conducted in and for the context of a real or realistic business entity. Teamwork and collaboration skills are fundamental to working in a business at operational and strategic management level.

However, it is important to ensure that group assessment is only used where it makes sense to use it (i.e. where it is appropriate for the learning outcome being assessed). It is also important that all group members have contributed to the assessment task to a degree where the assessor is sure of everyone’s ability in relation to the learning outcomes. Overusing group assessment makes it difficult to provide this assurance.

In many of the programmes monitored, assessments relied on the groups producing a shared product such as a report or a presentation. It was difficult to assess individual achievement of the learning outcomes in this context. There was no evidence showing which member of the groups produced which part of those assessments. Group work rather than group assessment may be more appropriate in these cases.

Two programmes that NZQA monitored used individual tasks within a group-based assessment, which were measured independently to ensure that learners had met all learning outcomes assessed. Moderators considered this a useful method because it provides learners with the opportunity to collaborate and achieve the learning outcomes.

Recording group presentations and using individual assessment forms (e.g. individual tasks mapped against each learning outcome being assessed) were also useful ways of collecting evidence for a group-based assessment.

Overall, NZQA recommends that group assessment is used conservatively in programmes leading to the award of the Level 5 and 6 Business qualifications. Instead, NZQA encourages TEOs to use a variety of assessment methods. These methods should be designed to reflect the learning outcomes and the context and requirements of a New Zealand business.

Conditions for assessment, resubmission and reassessment

NZQA also considered conditions of assessment, including resubmission and reassessment, as part of the programme monitoring. NZQA identified some issues which hindered learners from demonstrating their independent learning and may have prevented them from developing the capability to repeat their performance in an unfamiliar context.

Issues raised in this area included:

  • learners completing their assessments in class with excessive guidance from tutors
  • learners submitting drafts multiple times before submitting their assessments
  • TEOs allowing multiple opportunities for resubmission, particularly when learners received excessive feedback on the first attempt.

Authenticity

All TEOs delivering the Level 5 and 6 business programmes recognise that ensuring that learner work is authentic is critical to the quality assurance process. All TEOs monitored had policies and established practices for detecting plagiarism and the increasing use of synonym-replacement/text-rewriting software. Most programmes relied on plagiarism detection software as the first line of screening, and each tutor then assessed the similarity and relevance of the content in the learners’ work.

Learners who NZQA interviewed at monitoring visits understood plagiarism well, but NZQA still identified potential authenticity issues in learners’ work the TEO submitted for moderation. For example, NZQA moderators were unable to verify whether the submitted work was the learners’ own materials due to the overuse of unreferenced citations in their work.

NZQA also notes that ensuring learner work is authentic is more than just identifying plagiarism at the time learners submitted their work. For example, the issue of learners using synonym-replacement/text-rewriting software was specifically raised for three Level 5 and three Level 6 business programmes. Due to the careless use of that type of software, learner work lost meaning and became hard to comprehend.

A TEO’s policies and processes must include prevention, as well as detection, of a wide range of academic dishonesty.

For further information on preventing and detecting academic dishonesty please see the guide (PDF, 64KB) on the NZQA website.

Moderation systems and processes

All but two TEOs had relevant policies and processes for their internal and external moderation. However, NZQA found that the actual practice was often not effective.

Issues found in this area included:

  • TEO’s internal moderators failing to identify critical issues in assessments or assessor judgements, particularly issues that NZQA moderators identified (e.g. learning outcomes not being achieved by learners) 
  • TEO’s internal moderation reports containing limited feedback, or no reference to the learning outcomes or effective assessment design
  • TEOs not addressing issues identified by moderation
  • no follow-up moderation in response to assessments failing pre-assessment moderation
  • no evidence of TEOs conducting pre- or post-assessment moderation
  • lack of clear oversight of moderation system by TEO management
  • no plans of proposed actions to address issues identified by moderation. 

The best moderation systems ensured that:

  • assessment tools allowed learners to meet all learning outcomes
  • TEOs gathered feedback on the clarity of instructions and the appropriate level of assessment tasks
  • there was clear guidance for marking assessments
  • action plans arising from moderation reports were sufficiently detailed and monitored
  • moderation activities were clearly scheduled in a regular cycle
  • staff received professional development in assessment and moderation. 

Some TEOs had a dedicated moderation/quality assurance team or a specific staff member overseeing their moderation practice. These TEOs were able to provide a holistic picture of their assessment practice. NZQA found this was an effective way of managing moderation.

NZQA moderation

NZQA moderated a sample of marked learner work from each of the 20 Level 5 business programmes and 10 of the Level 6 business programmes NZQA monitored. For one Level 5 and one Level 6 programme, NZQA could not conduct valid moderation because monitoring identified that the TEO had made significant changes to the learning outcomes.

The purpose of NZQA’s moderation was to verify if learners the TEO had passed had supplied sufficient evidence to meet the learning outcomes being assessed.

Rates for agreement between the NZQA moderators and the TEO assessors ranged from 0 to 76 per cent for the Level 5 programmes monitored with an average of 21 per cent. Agreement rates ranged from 0 to 41 per cent for the Level 6 programmes monitored with an average of eight per cent.

The main reasons for not verifying assessor decisions include:

  • insufficient evidence that learner had met learning outcomes
  • learner work that lacked an understanding of the topic or requirements of the task
  • learner work not at the required level (e.g. learner work that was too basic, descriptive or did not apply knowledge as required to meet learning outcomes)
  • learner work with incorrect, irrelevant or superficial explanations, or inadequate research
  • learner work that relied heavily on referenced materials, and demonstrated little processing by the learner
  • irrelevant case studies used in assessments, especially when the New Zealand context was neglected
  • poor English that inhibited learners from demonstrating their understanding or meeting the requirements of the New Zealand business industry context
  • learner responses that were identical to other responses or the marking guide.

Poor assessment practice

Other issues of poor assessment practice noted by NZQA moderators include:

  • assessment tasks not allowing learners to meet approved learning outcomes
  • learners passing an assessment and/or module despite not meeting the minimum passing criteria for each learning outcome
  • marking guidance that was not detailed or clear enough to allow for consistent marking
  • assessors not following marking guidance
  • over-generous marking
  • assessors providing learners with little constructive feedback
  • potential issues with authenticity, (e.g. heavy reliance on published material or copying another learner’s work)
  • assessors not detecting learners’ use of synonym replacement software, which can make learners’ work meaningless.

Good assessment practice

Despite these many issues identified in this area, NZQA found some good assessment practices across the sector.

One good assessment practice instituted by a TEO was revising assessments before the next delivery to prevent incoming learners from accessing previous learner work. Another TEO mentioned that they were introducing alternative assessment sets for each course for the same reason.

Another positive move identified was that a TEO had implemented many actions to rectify issues identified in the 2017 NZQA moderation results. This TEO delivered both revised programmes, and the agreement rate for Level 5 was the fifth highest, and for Level 6 the highest agreement rate among all programmes monitored.

Many providers use case studies for students to apply their theoretical knowledge in simulated, real-world scenarios. One of the better examples of this was a TEO providing a clear, detailed, New Zealand case study that was relevant to the learning outcomes assessed. For example, a Human Resources course introduced a case study with a clear organisational chart, providing each staff member’s role expectations, along with detailed background. Because the provider used a New Zealand organisation, students were able to apply the relevant legislation. The NZQA moderator commented that the learners are provided with a realistic opportunity for how to implement New Zealand legislation in a human resource role.

In addition to being relevant to the New Zealand context, the best assessment materials:

  • provided learners with the opportunity to meet the learning outcomes
  • included a range of interesting and challenging assessment activities
  • progressively allowed learners to build on the previous learning outcome and/or assessment
  • resulted in articulate, thoughtful assignments or work-based application of skills that reflected the graduate profile.

As noted earlier, grades higher than a ‘C’ or ’50 per cent’ should mean that learners must achieve each learning outcome assessed. Some providers had this requirement clearly in their course outlines or programme documents. This was useful because it avoids any misunderstandings over marking for both assessors and learners.

Programme review

This section of the monitoring report addresses the ongoing review of the programme and the TEO’s self-assessment of the programme.

Monitoring found that this is the second weakest area next to assessment and moderation: only four Level 5 and three Level 6 programmes monitored met both criteria in this section.

Good practice suggests that TEOs review their programmes at least annually, though interim review activities may feed into the annual review. Reviews should be thorough and robust to ensure that the approved programme is still fit-for-purpose and meets the needs of stakeholders.

See Programme approval and accreditation rules 2018 criteria 4.1.7 and 6.1.4 for the aspects that need to be reviewed.

Issues with programme review

Programmes that did not meet the criteria for programme review failed to identify issues such as:

  • no appropriate engagement with stakeholders
  • superficial, sparse, or undetailed surveys or evaluative data
  • not analysing or using surveys or evaluative data for programme review
  • no action plans or proposed actions in response to issues identified in ongoing reviews
  • external advisory group only made up of internal academic staff members
  • no evidence of conducting external advisory group meetings, or only superficial evidence that did not outline actual discussions held at those meetings
  • gathering insufficient data to review whether graduates met the graduate profile of the qualifications
  • inadequate review leading to unsuitable changes to the programmes (e.g. lowering the levels of courses and materials).

Good ongoing review and self-review practice

During the monitoring, NZQA observed some good practices in this area across the sector.

These include:

  • collecting, analysing and using relevant data (e.g. success or retention rates)
  • conducting several types of surveys to gather evaluative information (e.g. course surveys, first impression surveys, and tutor surveys)
  • producing annual reports on programme evaluation to address specific Key Evaluative Questions (KEQs)
  • producing action plans for improvements with a timeframe to complete the outlined actions
  • providing detailed meeting minutes showing discussion with external stakeholders about learners’ employment prospects.

The annual programme review should be an opportunity to evaluate and ensure whether the programmes meet the qualification’s strategic purpose and outcome statements.

Programme structure and delivery

This section of the monitoring report addresses:

  • learning outcomes
  • delivery methods
  • programme length
  • learning hours.

Learning outcomes and delivery methods

There were cases found where TEOs had changed approved learning outcomes without notifying NZQA. While changes to programmes are a natural outcome of programme review (as over time learning outcomes may be refined, combined or not needed), NZQA must be notified of changes. This ensures the programme still allows the learner to meet the graduate profile. Three programmes monitored contained unapproved changes to the learning outcomes.

Monitoring did not find any significant issues in delivery methods.

Programme length and learning hours

The main issue identified was the lack of formal monitoring or structure for learners’ self-directed learning hours. Monitoring found that TEOs relied heavily on the quality of learner assessment work to measure how they had spent self-directed learning hours. Therefore, it was unclear how TEOs ensured learners were meeting the approved learning hours.

There were two TEOs that used a study-log, which required learners to record what they studied outside of class. This is one of the effective ways to monitor learners’ self-directed learning. Another good practice used by a TEO was posting activities or discussion topics on a learning management system (e.g. Moodle), aside from assessments, to encourage learner research or reading. Tutors also monitored learners’ engagement through the learning management system.

While it is important to have breaks in study sessions, TEOs should avoid long or excessively frequent breaks. Those breaks could mean the TEO is under-delivering the programmes, especially when there was no evidence of how the TEO is addressing potential gaps between the self-directed and directed learning hours.

One TEO delivered programmes in a compressed format, which NZQA discourages.

Other unapproved changes to programmes

Five Level 5 and four Level 6 business programmes included unapproved changes, such as:

  • adding a new course
  • changing the length of courses
  • changing assessment weightings.

TEOs must notify NZQA of any changes to an approved programme to ensure that it continues to meet approval and accreditation criteria.

Programme regulations

This section of the report addresses the regulations and how TEOs apply them for:

  • admission
  • credit recognition and transfer
  • recognition of prior learning
  • integration of practical and work-based components
  • normal progression within the programme.

Quality Management Systems and programme handbooks almost always provided sufficient policies and procedures in these areas. However, three TEOs failed to ensure that some of their international learners met English language proficiency requirements.

Issues with admission

All TEOs have clear, relevant, and appropriate regulations for admissions. However, three TEOs breached NZQF Programme Approval and Accreditation Rule 18, English language requirements for international learners.

Issues that NZQA identified include:

  • graduates from the Level 5 business programme not being required to provide further English proficiency evidence when proceeding internally to the Level 6 business programme, although the original evidence did not meet the requirements
  • TEO’s using their own internal English proficiency test after Rule 18 was amended to exclude this testing method as approved proof of English proficiency
  • learners admitted to the business programmes with unapproved English proficiency evidence (e.g. NCEA level 2 was accepted where the minimum requirement is higher).

NZQA recommends that TEOs monitor their admission processes regularly.

Credit recognition and transfer, and recognition of prior learning

Most TEOs had not used credit recognition and transfer (CRT) or recognition of prior learning (RPL) opportunities for the programmes monitored. Exceptions were for learners who transitioned from the old New Zealand Diploma in Business [Ref: 9003] or for learners transferring from discontinued programmes where NZQA requested such arrangements.

NZQA reviewed a sample of the evidence from five TEOs which have awarded CRT or RPL, aside from the cases noted above. All five TEOs had good evidence they had effectively applied their CRT/RPL processes.

Good practice for CRT/RPL is to gather sufficient evidence to show that learners have met the learning outcomes of the programme they are seeking recognition for.

Such evidence should include:

  • mapping the learning outcomes that the learner has previously achieved to the current programme learning outcomes, at the correct level on the NZQF
  • a supporting portfolio of work and/or verification of previous achievements by professional conversation/learner interview.

All acceptable evidence should be held in the learner’s academic file.

See further guidance at recognising learning and awarding credit.

Integration of practical and work-based components

Both the Level 5 and 6 qualifications [Refs: 2459 and 2460] include the following condition: “Programme design and delivery, and all assessment will be conducted in the context of a real or realistic business entity, and in light of the requirements of that context.“

NZQA found that the programmes monitored were predominantly theoretical rather than applying the theories learned in a real or realistic context. For example, many TEOs were preparing learners for further study (e.g. Level 7 business related programmes), which is not the main or only purpose of the qualifications. NZQA recommends that some TEOs re-evaluate the purpose of their programme to ensure that it aligns with the qualification conditions.

One TEO offered learners internship opportunities at its group organisation so that learners could apply the theories learned in class to the real context. Another TEO had embedded work-integrated learning into the programme. While these examples are good practice and ensure that learners were exposed to real business entities, these opportunities must clearly align with learning outcomes assessed in the relevant courses.

Some TEOs invited guest speakers and/or introduced field trips to local businesses. Those activities appeared effective and were often welcomed by learners, especially when they covered topics learners considered relevant.

Other TEOs used New Zealand-based case studies to apply theories taught in class in realistic business situations. Although using international case studies is acceptable where relevant, international case studies should not be used where learning outcomes aim explicitly at a New Zealand context.

Normal progression within the programme

All programmes clearly indicated compulsory and strand-specific courses and the prerequisites or co-requisite for some courses if relevant.

Resources

This section of the monitoring report addresses:

  • academic staffing and professional development
  • teaching or learning facilities
  • educational resources
  • learner support.

Academic staffing and professional development

Monitors found that most academic staff were qualified to teach in the relevant subject areas. However, it was not always clear how academic staff maintained their expertise in and current knowledge of the subject.

While staff had research and further study activities in their professional development, New Zealand industry-specific engagement was often absent (e.g. being a member of a relevant professional organisation or attending a business-specific forum).

Monitors also found that many academic staff did not have, nor were they studying towards, an adult teaching qualification. Although this is not outlined explicitly in the rules, it is important that academic staff members should be properly trained in education or adult education, especially when NZQA identified problems in assessment and moderation. NZQA considers this type of professional development would help TEOs improve in that area.

Many TEOs conduct staff forums, workshops, or staff symposia to discuss teaching issues. These issues could include moderation practice, fair marking, and plagiarism detection. This is a positive example of professional development found during the monitoring period.

Providing learner support

Resourcing of learner support was consistently good across all programmes NZQA monitored.

TEOs supported learners before they started the programme, offered an orientation programme for a few days or week at the start of the programme and provided ongoing pastoral support for learners.

Some TEOs also offered academic writing, English language support and career guidance to learners outside of the programme requirements.

Providing appropriate facilities and resources

Teaching and learning facilities and education resources were also appropriate for the programmes NZQA monitored.

NZQA notes that TEOs are increasingly using online learning management systems (LMS), such as Moodle, to facilitate learning in and out of class.

An LMS can be used to:

  • share learning resources
  • facilitate learning activities (such as quizzes and discussion forums)
  • track learner participation and engagement.

Top tips from the 2018 Levels 5 and 6 business programmes monitoring activities

  1. Provide professional development opportunities for academic staff and internal moderators to improve their skills in the areas of assessment design, assessment marking, and moderation.
  2. Ensure that the focus is on learning outcomes and NZQF level when designing assessments, quality assuring assessment materials and marking learner work.
  3. Implement a system to oversee moderation processes that gives a clear picture of identified issues and how the TEO is addressing those issues.
  4. Conduct a robust programme review to critically evaluate whether the programme meets the strategic purpose and graduate profile of the qualification.
  5. Improve consultation with external stakeholders (e.g. local advisory group) by creating more opportunities to get constructive feedback on the programme.
  6. Encourage academic staff to engage with the New Zealand business industry to ensure the programmes are relevant and teach skills currently in demand.
  7. Apply a broad range of prevention strategies to maintain learners’ academic honesty.
  8. Provide learners with further academic support outside of the notional learning hours to ensure graduates from the programmes meet industry needs.

 Back to summaries of monitoring findings

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz