How to get the most out of an external audit

An audit should add value to the organisation being audited. It is an opportunity for an independent external body to validate the effectiveness of internal systems and processes. It also allows for constructive feedback on what is being done well and areas for improvement.

An external audit should not be a one-off event requiring extensive preparation. However, some pre-audit preparation will enable the process to run smoothly.

Pre-audit documents

The audit notification letter requests specific documents to be sent to NZQA before the audit. Timely receipt of these documents enables the auditor to accurately scope the audit and to perform a detailed analysis. The auditor becomes familiar with the organisation and its systems before the visit. This saves valuable audit time. The auditor can then focus on the collection of evidence to support findings of good practice as well as areas for improvement. The documents required are:

• Copy of the current quality management system (QMS)
• Provider Update Form (sent with the audit notification letter)
• Self-review documents for private training establishments, government training establishments and wananga; industry training organisations; course owners; Adult and Community Education providers.

Quality management system (QMS)

The focus of any audit is to assess how well the QMS reflects the size, nature and complexity of the organisation. It also assesses how well the QMS is being implemented and the impact it has on learning outcomes for students. Refer to the section on developing and improving a QMS for a fuller explanation.

Self-review and evaluation

An external audit should validate that the organisation has robust internal self-review systems which contribute to positive learning outcomes for students. Self-review usually has two components.

Internal audit is a control function that aims primarily to determine compliance with policy and procedures.

Review is an assurance function. It ensures that the currency and effectiveness of procedures, and their relevance to and impact on other parts of the QMS, are managed.

Self-review or evaluation covers a number of points:

• identification of key strengths and weaknesses
• actions taken to improve performance
• the effectiveness of policies and procedures in meeting goals and objectives
• the learning experience and final outcomes for learners
• whether policies and procedures cover all of the standard and any other relevant standards. Examples are the Code of Practice and Accreditation and Moderation Action Plan requirements
• whether actual practice complies with the QMS.

As mentioned above, policies and procedures should cover the requirements of the standard. If practices comply with policies and procedures, it follows that practices should comply with the standard.

There are many ways to conduct effective self-review. What is important is that review processes are planned, robust and objective, and where shortcomings are identified, action is taken to address them.

Consider the following points when planning a review programme:

• how long the procedure has been in existence; new procedures may require more frequent review until they are well established
• the degree of complexity of tasks involved
• the level of exposure to risk if procedures are not accurate or being applied consistently
• the link between processes and measurement of goals and objectives. Will the review provide evidence of achievement of your goals and objectives?

It is usually helpful to begin with an internal audit which asks the question: "Are we implementing and complying with our policies and procedures?" The results of the audit will indicate whether policies and procedures need revising and will also provide reliable base data for any review that follows.

Self-evaluation

NZQA has self-evaluation workbooks associated with the various quality standards. The workbooks can assist TEOs to evaluate their performance. However, it is expected that planned and systematic internal audit and review processes are conducted as part of the QMS.

The self-evaluation may be sent to NZQA as set out in the workbook or in any other way. The document should clearly show any findings made and identify the evidence that supports those findings. It should also outline plans for improvements. If used, the Self-Evaluation Guide and Workbook should be used in a way that adds value to education and training. For example, simply writing "yes" next to each requirement of QA Standard One does not provide helpful information to the organisation. Nor does merely cross-referencing the requirements to the QMS policies and procedures

Meeting requirements through good practice

One way of conducting successful self-review is to work systematically through each requirement of the standard. This may include the Code of Practice and other relevant standards. Then ask the question: "How can I show that I am meeting this requirement?"

An audit relies on positive evidence that the organisation is complying with requirements. Evidence must be tangible, not anecdotal, and there must be evidence that each requirement is being met. As mentioned in the planning section, the auditor uses triangulation (three different sources of evidence) to ensure that there is sufficient evidence to substantiate a finding.

When conducting self-review ask whether there is sufficient evidence that policies and procedures are being implemented effectively. They should also result in effective teaching and learning in the organisation.

The same audit principles apply irrespective of whether it is an internal audit or an external audit. There must be evidence and the auditor must be able to triangulate the evidence. The reason for forwarding the QMS and self-review before the audit is so that NZQA can:

• understand the business
• evaluate the rigour of the QMS and self-review practices
• verify the results of self-review.

Remember, the more effective the self-review, the better the organisation will perform and the more value you will receive from the audit.

Page updated: 07 August 2007