Assessment case study

1 ElectroTechnology Industry Training Organisation (ETITO)

11815 Answer customer enquiries on the telephone in a wide range of contexts (Level 3)

Case studies of assessment and moderation
Commentary
Background
Issues encountered
Quality systems
Support documents

Commentary

This is an example of contexualised assessment that is based on naturally occurring evidence and grounded in on-job performance. Its stated aim is to be user friendly, simple and clear, and structured so it enabled assessors to carry out their assessments during normal workplace activities. It has been used extensively and verification from stakeholders has been uniformly positive. The process could be readily adapted to other in-house situations.

Key points to note in the support documents are:

• The Assessment Plan is holistic and focuses on evidence required rather than on the unit standard.
• The Evidence Guide draws on naturally occurring evidence. The example is an illustration of standards being aligned with initial certification and on-going performance management. The combined assessment of Elements 2 and 3 was done because it made sense. However, such combination may not be appropriate in all contexts, or in the future.
• The Observation Checklist collects evidence at the PC level. It would probably be preferable for prominence to be given to the Element in the Observation Checklist documents. However it is interesting that this method is used as a form of guidance notes to the assessor aboutwhat constitutes acceptable performance in the industry rather than using the PC to generate tasks. It also demonstrates that the PCs have been considered in making the judgement at element level.
• Question Bank/Discussion section is also useful guidance to the assessor. It makes allowance for alternative answers and these are recorded - probably for moderation purposes. Question Banks provide an extra source of evidence for the assessor to ensure coverage of the full requirements of the unit standard without being too prescriptive. They allow for the professional judgement of the assessor about which gaps in the other evidence need to be filled to ensure sufficiency. It also offers triangulation of the evidence to ensure authenticity.

Background

The assessment guide presented here was developed by an expert working panel of call centre industry representatives and assessors who were involved in the development of the National Certificate in Call Centre Operations, Level 3. This group worked collaboratively with ETITO staff who assisted with technical advice on what the assessment guide needed to cover, how to structure and format it, and administrative support.

The assessment guide was designed for use in the workplace setting. It is used to assess employees on job, drawing on naturally occurring events and activities. The guide needed to be user friendly, simple and clear, and structured so it enabled assessors to carry out their assessments during normal workplace activities.

In the call centre industry, companies commonly use in house assessment tools to evaluate the performance of their staff on an ongoing basis. This means the assessors are already familiar with observing performance against company standards, and generally know what their companies expect of staff. Assessing against the national qualifications is essentially an extension to this. This particular feature of assessment in the call centre sector is not common to all ETITO sectors.

Issues encountered

The elements and performance criteria of the unit standard are not written in the same order as activities are carried out in the workplace. For example, element 2 seems to suggest that analysing the customers needs and expectations occurs separately from identifying and meeting those expectations. In reality, there is no great distinction between these stages during a normal interaction with a customer on the telephone. As the candidate identifies the customers needs, s/he is also analysing and working out how to meet the customer's needs. The solution to this issue was to combine the assessment of elements 2 and 3.

Workplace assessors in call centres cannot control the types of customer enquiries that will come in a day when they are observing the candidates performance. It is difficult to catch the moments when a candidate might be dealing with customers who are under stress, or who have special requirements. ETITO decided it was more important that assessors used naturally occurring evidence, rather than simulation, so the group chose to use a series of questions to cover some aspects of element 1.

As part of the development process, members of the working group used the evidence guides in a trial run. This resulted in some minor adjustments.

The assessment guide for 11815 has also been used extensively in ETITO assessor training courses. Feedback from the trainee assessors has been consistently positive. They find the guide easy to follow, and are able to come up with consistent assessment decisions using it.

Key learning points to note are:

Structure assessment around natural work activity

This assessment has been designed to follow the normal work process, rather than the order of the elements and the performance criteria in unit standard 11815. Natural observation activities for this unit combine a number of elements and performance criteria

Use questions to cover aspects of unit standards which are not easy to observe

The assessment draws on naturally occurring evidence, rather than simulation. Workplace Assessors will not always be observing when the less common events occur in the workplace. Assessors to come to an informed decision using the whole picture of evidence along with questioning on other situations.

Industry people are the best people to identify relevant assessment evidence which can be used in their industry context

The industry working group who worked on this guide quickly identified that the evidence required for unit 11815 fitted well with the call observation activities they already undertook in their workplaces. The evidence guide development naturally flowed on from this.

Develop an assessment tool that fits the sector

Call centre assessors want to see their own company requirements covered in the assessments, and they are happy to identify these and include them as part of the assessment. They are also comfortable making decisions about how many times they need to observe a candidate. In some ETITO sectors there is a need to provide more guidance for the assessors, especially where these sectors have not had a long history of assessment. Some of the industry sectors are used to using "tick boxes" and want these. The main message is to create assessment materials in a format the assessors are comfortable working with

Quality systems

In order to gain their registration, assessors have to achieve Unit Standard 4098 Assess candidate performance against standards, and also are required to gain the qualification they will be assessing. In most cases, assessors will complete their assessor training during a course arranged by ETITO. The training provider delivers a customised assessor training course, using ETITO assessment material and examples, which are relevant to the trainee assessors' industry context.

Once registered, assessors are expected to use the assessment guides whenever they carry out an assessment.

The quality systems are critically important to the success of the assessment materials.

1. ETITO asks organisations to nominate people with expertise in their role to be assessors.
2. ETITO trains assessors to unit standard 4098 in industry specific assessor training courses.
3. Assessors must themselves have the skills and knowledge they will in turn assess. Call centre assessors are required to hold the qualification(s) they assess.
4. All registered ETITO workplace assessors use the industry developed assessment guides.
5. Assessors are required to provide samples of their assessment for internal moderation on a regular basis.

The relevant ETITO advisory group owns all ETITO assessment guides. They are subject to regular review. ETITO's Assessment Manager oversees the implementation of any changes required.

Support documents

The following documents are provided here as examples of assessment in practice but are not intended as definitive guides. They are available to view in Word and PDF and include:

• Assessment plan
• Evidence guide
• Observation checklist
• Question bank/discussion

Page updated: 17 December 2002