Overview
A key resource is the availability and time of clinician assessors.
In this lesson, we turn to the ‘behind the scenes’ components of the assessment process. These are the underpinning or support components – the resources. Appropriate and adequate resources are crucial to the success of the assessment process. They contribute significantly to the functioning of a defensible assessment system and are key factors in ensuring that the assessment system is sustainable.
Resource requirements
Different environments call for differences in implementing assessment processes; and it is expected that resources and support structures will vary across assessment sites. However, practical experience indicates that the resources outlined below are needed to ensure a defensible and sustainable workplace-based assessment system.
Leadership
Visible commitment from leadership is essential.
- Strong and committed leadership gives workplace-based assessment a head-start towards successful implementation.
- Workplace-based assessment requires commitment and ongoing support from key personnel in clinical, organisational and administrative positions.39
Staffing
- Staff involved in workplace-based assessment need to have clearly-defined roles and responsibilities, with adequate time allocated for the performance of these tasks.
- Assessment is a team effort; each individual needs to understand their role and its purpose in the assessment process.
- Staff training may be necessary; and ongoing professional development for staff will ensure that the program proceeds in a constructive way.
- The assessment team needs strong leadership, with the support of senior clinicians and management.
IMG orientation
- IMG orientation to workplace-based assessment is necessary to ensure the objectives of the assessment are understood and there is a commitment to the entire assessment process.
Administrative system
A functional workplace-based assessment system requires dedicated staff support.
- Administrative duties include the scheduling of assessments, the arrangement of IMG-patient encounters and the provision and collection of forms for documenting assessment processes and patient consent.
- Post-assessment, forms need to be collected and the information entered and stored in electronic form.
- Other administrative functions include communication within the workplace, and arranging consultation with stakeholders.
Data storage
- A spreadsheet for storing and managing data should meet the specific needs of the assessment processes and AMC/medical board reporting requirements.
- Assessment data must be entered and recorded accurately, and in a format which enables appropriate analysis and reporting. Staff training for these functions may be necessary.
- Data management and access must adhere to privacy and security regulations.
- Data stored on this system will inform decisions about the registration of IMGs, and provide the evidence required for possible remediation, appeals, consideration of indemnity issues.
- Data stored on this system will enable the assessing authority to keep track of the IMG’s progress, and use the information in its regular evaluation of the assessment program.
Quality control
Assessment data must provide reliable indicators that serve as a basis for valid inferences about the ability to practise independently.
- As a high-stakes assessment, workplace-based assessment depends on the integrity and fidelity of its processes and systems. Assessment data on IMGs must provide reliable indicators of performance, and serve as a basis for valid inferences about their ability to practice independently.
- Assessment data on IMGs provide evidence about the implementation of the assessment program and form the basis for reviewing, monitoring and reporting on the processes and systems implemented.
- Regular review and necessary updating of the assessment processes and systems are important in maintaining sustainability and accountability.
Budgetary considerations
- IMGs will be required to pay a fee to undertake workplace-based assessment.
- The budget should address expected income requirements, anticipated income sources and expenditure estimates. Expenditure estimates should include both the expected costs of the assessment activities and the administrative support required.
- Set-up costs need to be calculated and their funding sources identified, as these costs may be incurred before income from the assessment program is received.
- An allocation of funding to the ongoing maintenance and improvement of the assessment program is necessary to ensure that the program is sustainable.
Educational expertise
Authorities conducting workplace-based assessment should consider drawing regularly on educational expertise and experience. Guidance from an educator might be sought in the following areas:
- To summarise findings from the medical education literature in regard to the reliability of the assessment methods selected; and in particular to indicate the number of encounters and time required to give confidence that the methods to be used are acceptably reliable;
- To provide information on validated methods of assessment that can be applied to assessment within any given clinical setting;
- To give advice, based on experience, on feasibility and practical aspects of implementing performance-based assessment;
- To give advice on data collection, storage and analysis; and to suggest ways that these data might contribute to review and renewal of the assessment process;
- To provide leadership and/or support in training assessors for their roles in assessing and giving feedback;
- To provide leadership and/or support in monitoring the sustainability of the assessment program;
- To provide information on evidence-based findings on assessment practices;
- To assist in developing and articulating educational policies relevant to assessment.
References:
39 op. cit. Page G. #4.