This course provides background information about the principles that underpin effective assessment and how those principles apply to assessment of clinical performance conducted within the workplace. It is designed to complement the Australian Medical Council (AMC) workplace-based assessment (WBA) accreditation guidelines and procedures for international medical graduates (IMGs). However, the information contained in this course is generally applicable to any health profession introducing WBA.
Workplace-based assessment – what is it?
Workplace-based assessment encourages IMGs to attain the abilities and attitudes required for safe and independent practice in Australian clinical settings.
Workplace-based assessment (WBA) is a form of authentic assessment; it tests performance in the real environment facing doctors in their everyday clinical practice. WBA should enable IMGs to demonstrate their progress in integrating clinical knowledge and skill as a basis for effective clinical judgments and decisions. It should also track their development towards becoming an independent, self-monitoring clinical practitioner in the Australian setting, which includes their use of effective communication, respect for patients, working productively within a team of healthcare professionals, and applying risk-management skills.
How this form of assessment relates to other forms of assessment is clarified by reference to Miller’s Pyramid. In 1990 Miller used a pyramid to provide a framework for clinical assessment which is shown at Figure 1.1 Relative to the levels of assessment in Miller’s Pyramid, written examinations assess candidates at the ‘knows’ and ‘knows how’ levels, and clinical assessments assess at the ‘shows how’ and ‘does’ levels. WBA is an assessment at the ‘does’ level, as its focus is on performance in the workplace setting.
In the context of the current AMC examinations used to assess IMGs, the multiple choice question (MCQ) examination is at the ‘knows how’ level of the pyramid, since the examination in large part presents clinical vignette questions which test clinical decision-making skills – the IMGs ability to apply knowledge to clinical problems.
The clinical examination conducted by the AMC is in an Objective Structured Clinical Examination (OSCE) format. In this examination, candidates rotate through a series of eight-minute stations where they are asked to perform a task during an encounter with either a real or simulated patient. It is at the ‘shows how’ level, as it assesses candidates’ ability to perform in simulated situations.
A strength of workplace-based assessment is that it measures what a doctor actually does in practice.
While OSCEs assess competence, that is, what a candidate is capable of doing within controlled environments, WBA measures what a doctor actually does in practice.2, 3 WBA contributes important information about an IMG’s overall suitability for independent practice in Australia.
Objectives of workplace-based assessment
The aim of a summative assessment, such as the AMC examination, is to ensure that an IMG possess an adequate set of clinical skills and other essential attributes to practice safely within the Australian health care environment and cultural setting. In the case of the AMC, workplace-based assessmentis used to:
- Monitor an IMG’s progress through the required supervised pathway and signify satisfactory completion of the program;
- Provide guidance for specific learning needs of IMGs; and
- Determine whether and when IMGs are ready to proceed to independent (or more independent) practice in Australia.
Principles underpinning workplace-based assessment
Workplace-based assessment involves the measurement of abilities and attitudes within highly complex areas of activity. To be effective, performance in a complex area of activity requires focussed observation of IMGs in clinical practice by assessors drawn from a team of clinicians who are experienced in relevant clinical practice and trained in assessment of IMGs and other medical trainees and practitioners.
Given the high stakes for the IMGs and the community, workplace-based assessment, must have established reliability and validity to ensure that defensible decisions are made. Workplace-based assessment should assess performance across a prescribed range of clinical areas and dimensions of clinical practice.
The selection of assessment methods is critical to the success of workplace-based assessment. The number and nature of the tasks and the total time involved have been shown to be key factors underpinning the reliability and validity of workplace-based assessment. The following should be considered in developing a suitable assessment program:
- Clinical performance is multifaceted and no single method can assess all the elements that contribute to good medical practice. In addition, assessment methods vary in their effectiveness to assess across different elements of performance. Accordingly, it is essential to use a variety of assessment methods that complement each other in the elements they assess.
- To ensure that an assessment decision is reliable, the assessment must be planned to sample across a variety of clinical areas. It is recommended that multiple observations are used in various clinical settings over a period of time.
- Since assessors may vary in their tendency to be harsh or lenient in scoring a candidate, the best method to moderate this tendency is to ensure that the assessment of an individual candidate is based on the opinion of trained multiple assessors.
Clinical dimensions for workplace-based assessment
The focus of workplace-based assessment should be on the candidate’s application of their knowledge and clinical skills in their clinical work. A successful workplace-based assessment program should assess the following aspects of performance:
- Clinical skills, including history taking, physical examinations, investigations and diagnosis, prescribing, management, clinical procedures and counselling/patient education.
- Clinical judgement
- Communication skills
- Ability to work as an effective member of the health care team
- Ability to9 apply aspects of public health relevant to clinical settings
- Cultural competence
- Professionalism
- Attention to patient safety
Assessment methods used for workplace-based assessment
Effective workplace-based assessment should incorporate several assessment strategies, including both direct observations of in-patent encounters and indirect methods to assess other clinical dimensions. Strategies that nay be used for workplace-based assessment include:
- Mini-Clinical Evaluation Exercise (min-CEX)
- Direct Observation of Procedural Skills (DOPS)
- Case-based Discussion (CBD)
- In-Training Assessment (ITA)/Structured Supervision Reports
- Multi-Source Feedback/360 Degree Assessment
Choice and design of assessment methods with clearly described criteria and assessor training are essential elements of a successful workplace based assessment program.
How to use workplace-based assessment in the Standard Pathway
IMGs who have obtained qualifications from authorities that are currently not designated as competent authorities are required to undertake a mandatory screening MCQ examination (on-shore or off-shore) followed by further assessment in either the Standard Pathway (AMC Examinations) or the Standard Pathway (workplace-based assessment). Both of these pathways lead to the award of the AMC Certificate and eligibility for general registration.
The Standard Pathway (AMC Examination) has been available for some time. In this pathway, IMGs present for an MCQ examination and then a clinical examination to qualify for the AMC certificate. Medical boards then require twelve months of supervised practice prior to general registration.
An alternative pathway, the Standard Pathway (workplace-based assessment), is available for IMGs who are not specialists and who are not eligible for the Competent Authority (CA) Pathway and who are currently appointed to a hospital or general practice position.
The goal of the Standard Pathway (workplace-based assessment) is to ensure that an IMG possesses an adequate and appropriate set of clinical skills and other essential attributes to practise safely within the Australian health care environment and cultural setting. Workplace-based assessment is additional to normal supervision requirements that apply to all IMGs and doctors-in-training, and is a more comprehensive assessment than that required for the CA Pathway.
The opportunity to conduct the assessment over a sustained period allows a more comprehensive assessment of clinical competency and performance than has been possible before.
Given the high stakes for the IMG and the community, workplace-based assessments must have established reliability and validity to ensure defensible decisions are made. Workplace-based assessment should assess performance across a prescribed range of clinical areas and dimensions of clinical practice. This course addresses these requirements.
The opportunity to conduct the assessment over a prolonged period as part of the Standard Pathway allows a more comprehensive assessment of clinical competency and performance than has been possible before.
References
1 Miller G. The assessment of clinical skills/competence/performance. Acad Med 1990;65:S63-S67.
2 Rethans JJ, Norcini JJ, Baron-Maldonado M, Blackmore D, Jolly BC, LaDuca T, Lew S, Page GC, Southgate LH. The relationship between competence and performance: implications for assessing practice performance. Med Educ 2002;36:901-909.
3 Van der Vleuten CPM. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ 1996;1:41-67.