Policy and Guidelines for the Conduct of External Evaluation and Review

• Introduction
• Key Components
• Undertaking External Evaluation and Review
• Stage One: Developing the Scope and Plan for an External Evaluation and Review
• Stage Two: Undertaking the On-Site Enquiry
• Stage Three: Reaching Judgements on Educational Performance and Capability in Self-Assessment
• Stage Four: The External Evaluation and Review Report
• Appendix One: Principles Guiding the Conduct of External Evaluation and Reviews
• Appendix Two: How the Key Evaluation Questions Relate to the Tertiary Evaluation Indicators
• Appendix Three: A Step-By-Step Guide to Using the Performance Criteria Rubrics
• Appendix Four: Generic Example of Developing a Plan of Enquiry

Introduction

This resource sets out the New Zealand Qualifications Authority (NZQA) policies, criteria and procedures for the external evaluation and review of private training establishments (PTEs), wananga and institutes of technology and polytechnics.

The information in this resource is also available as a dowloadable pdf: Policy and Guidelines for the Conduct of External Evaluation and Review (PDF, 222KB).

Rubric descriptors revised

April 2012: Changes to some of the rubric descriptors were made to ensure a sharper separation between higher and lower ratings. The revised rubrics (PDF, 17KB) replace those listed within this resource and the above downloadable pdf. 

The Legislative and Policy Environment

Self-assessment and external evaluation and review are requirements of course approval and accreditation (under sections 258 & 259 of the Education Act 1989) for all tertiary education organisations (TEOs) that are entitled to apply. The requirements are set through the course approval and accreditation criteria and policies established by NZQA under section 253(1)(d) & (e) of the Act.

The criteria and policies for private training establishment registration require self-assessment and external evaluation and review at an organisational level, in addition to the individual courses they own or provide. These criteria and policies are set by NZQA under section 253(1)(ca) of the Act.

NZQA is responsible for ensuring TEOs continue to comply with the policies and criteria after initial approval and accreditation of courses and/or registration is granted. However, the New Zealand Vice-Chancellors' Committee (NZVCC) has statutory responsibility for university compliance, and the Institutes of Technology and Polytechnic Quality (ITPQ) is responsible, under delegated authority from NZQA, for the polytechnic sector compliance.

The Quality Assurance Framework

These policies and criteria are gazetted under the Education Act 1989. They provide for a quality assurance framework established by NZQA and comprising four components:

• initial entry processes of course approval, accreditation to deliver a course, registration of private training establishments and accreditation to assess standards on the National Qualifications Framework
• self-assessment
• external evaluation and review
• managing the consequences of poor performance.

The quality assurance framework uses the theory and practice of evaluation in order to focus on the quality of educational outcomes. Evaluation as a discipline provides a systematic basis for answering questions about quality and value. Its use in tertiary education provides a practical approach to focusing on outcomes and key contributing processes, using both qualitative and quantitative information.

Evaluative quality assurance draws on a wide range of international theory and practice, in particular mixed method and participatory approaches, to arrive at a robust process for reaching consistent and comparable judgements.

The following four principles underpin the quality assurance framework.

"High Trust and High Accountability"

Evaluative quality assurance operates in an environment of "high trust and high accountability" throughout the tertiary sector. The relationship between government agencies and TEOs is based on good communication and collaboration.

A high trust environment recognises a TEO has autonomy over its own processes, with minimal Crown intervention, except in relation to undertaking its regulatory roles.

The government recognises that quality is the direct responsibility of each individual TEO and that the TEO is accountable for the educational outcomes it achieves and the ongoing improvement of educational performance.

In return for greater autonomy, the Crown needs to have a high level of confidence in each TEO. External evaluation and review is the mechanism that will determine those levels of confidence. However, if the performance of a TEO falls below an acceptable level, that organisation will be monitored more closely by central agencies.

Quality As A Dynamic Concept

Quality is dynamic and will look different in different contexts. In tertiary education, what matters is the value that learners gain from their learning experience, the utility of their qualifications and the extent to which positive, longer-term outcomes occur.

This dynamic concept of quality is consistent with international trends1 where there is a shift from quality control (meeting input standards) to quality enhancement and striving for excellence. This approach emphasises the aspects of an education experience that generate better outcomes.

Every TEO will have an understanding of quality which varies according to the purpose and goals of the organisation. In the TEO context, quality relates to how well learners actually achieve and the extent to which that achievement meets both their needs and those of the wide range of groups and individuals that have a direct, formal and often economic interest in the work of TEOs.

A Focus On Outcomes

Traditional quality assurance has focused mainly on inputs, systems and activities. This assumes that doing these well provides an assurance of quality.

Evaluative quality assurance is focused on the outcomes of tertiary education and the key processes that contribute to these outcomes. A key difference from traditional approaches is that while evaluative quality assurance also looks at processes, it does so from the perspective of the utility or impact of these processes on what is done and achieved - the "valued outcomes" of tertiary education.

Outcomes are affected by contextual inputs such as student characteristics, staff and resources. An important indicator of quality in practice is a TEO taking all reasonable steps to maximise learner achievement.

Distinctive Contribution

Evaluative quality assurance is flexible and designed to constructively respond to the distinctive contributions and character of the wide range of tertiary education organisations in New Zealand.

Like the conception of quality, it recognises that evaluation will look different in different contexts, while still retaining the required consistency of approach to evaluate performance credibly.

In particular, the approach to evaluation taken by the distinctively Maori organisations, notably Wananga and Maori private training establishments, will directly reflect their values, beliefs and aspirations.