2.2 Completing observed assessments
- For observed assessments, trainees are observed and assessed in the workplace undertaking routine activities and interactions required for post-programme practice.
- As observed assessments assess learning from a module, trainees should be sufficiently comfortable with the module content to allow them to draw on the full range of skills and applied knowledge when they attempt an observed assessment. Trainees should have received training and had an opportunity to practise the activity or interaction before being assessed and should be ready for associated training activities to be completed.
- Trainees should be supported to complete observed assessments to meet the progression requirements of the programme. The time taken to arrange and undertake observed assessments should be considered and planned for to enable trainees to meet their progression requirements.
- It is good practice for trainees and training officers to discuss the trainee’s readiness to undertake an observed assessment before arranging an assessment.
- One observed assessment can be submitted for one DOPS or OCE only. A single assessment cannot be mapped to multiple DOPS or OCEs in the e-portfolio.
- Any reasonable adjustments made to support trainees to complete tasks in the workplace should also be applied when the trainee is undertaking the task for an observed assessment.
2.2.1 Preparing for an observed assessment
- The activities and interactions to be undertaken for DOPS and OCEs for each module are specified in the curriculum. There are a range of DOPS and OCEs which can be completed for each module. The training officer and trainee should discuss which observed assessments the trainee will complete. In doing so, they should aim to showcase a range of skills across each module.
- A trainee’s portfolio must include both DOPS and OCEs.
- Each specialty’s observed assessments were written by experienced healthcare scientists to be representative of routine activities and interactions required for post-programme practice. It is therefore important that trainees are assessed across this range of activities and interactions to ensure they have the skills, behaviours, and attitudes to be able to engage in the full range of practice after the programme.
- Observed assessments and their associated marking criteria reflect expectations for the trainee’s stage of practice.
- Rotation modules include training activities which invite reflection on observations made in the workplace at a time when trainees have no or limited hands-on experience of practice. Observed assessments for rotation modules are therefore simple activities or interactions which can be completed by trainees based on their observations and on knowledge from the academic component of the programme, supported by some training in the workplace. Trainees must pass the required number of observed assessments for rotation modules and should be supported appropriately where reattempts are required. Feedback from observed assessments for rotation modules can be useful to identify training needs and guide a plan for training for subsequent phases.
- Wherever possible the activity or interaction being assessed should be observed as part of the trainee’s everyday practice. This ensures the trainee can perform the activity or interaction in a real-world situation including the associated challenges and complications that may include. The aim of observed assessment in the workplace is to assess the trainee’s skills, behaviours and attitudes as applied authentically in practice, rather than ensuring trainees can provide a textbook response in a controlled environment.
- The observed activity or interaction may not go to plan, but the trainee may still be able to demonstrate the appropriate skills, behaviours, and attitudes to successfully complete the assessment by ensuring the most appropriate outcome.
- Where patients are involved, appropriate consent should be sought, and consideration given to how the patient might be involved in the assessment.
- The activity or interaction can be simulated where necessary and the simulation should seek to replicate the realities of practice as authentically as possible. It is not sufficient for the trainee to provide a verbal ‘walk through’ of the activity or interaction, or evidence that it has been completed on another occasion. The trainee must perform the activity or interaction in front of an assessor.
- If a simulated activity or interaction requires an ‘actor’ to role-play the part of another person involved, they must interact with the trainee ‘in role only’. The actor should provide no prompting or guidance to the trainee or assessor. Assessors should not play a role in simulations to ensure that they are able to independently observe the trainee’s performance in full.
Because the goal of training and assessment is to produce physicians who can practise in the clinical setting unsupervised, learners should be observed in the setting in which they need to demonstrate clinical competence. Actual clinical encounters are often more complex and nuanced than simulations or role plays and involve variable context; direct observation of actual clinical care enables observation of the clinical skills required to navigate this complexity
Kogan, J.R., Hatala, R., Hauer, K.E. et al. Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education. Perspect Med Educ 6, 286–305 (2017). https://doi.org/10.1007/s40037-017-0376-7
2.2.2 The observed assessment
- There is no set amount of time specified for completing an observed assessment. The assessment should take as long as is reasonably required to complete the activity or interaction fully. As a rule of thumb, most observed assessments should take around 15 minutes to perform.
- It is expected that a single assessor will be sufficient for most observed assessments. A maximum of two assessors can assess an observed assessment.
- Assessors must only observe during the activity or interaction. Assessors must not prompt or guide trainees, ask questions or enter a discussion with the trainee while the activity or interaction is taking place.
- After the trainee has completed the activity or interaction required for the assessment, the assessor may spend up to 10 minutes asking the trainee questions about the activity or interaction.
- All trainee involvement and input required to complete the activity or interaction specified should be observed. Most observed assessments should be completed as a single observation from start to finish. For some activities there may be intervals which do not need observation, such as automated phases which do not required user input.
- For specialty and core observed assessments, trainees may not refer to supporting resources, such as notes or aids specifically designed to help them perform the task or pass the assessment. They may refer to resources reasonably expected to be present when completing the activity or interaction in routine practice.
- It is good practice to involve different assessors across the required number of observed assessments to get a range of opinions and feedback on practice, particularly for specialist areas of practice where assessors with specialist knowledge may be required.
- A trainees may request that their training officer, or another person involved in their training, is with them during the assessment for support. This person’s presence at the assessment is only to facilitate the trainee’s engagement with the assessment, they may not contribute to the trainee’s responses and may not contest the outcome of the assessment determined by the assessor.
- To facilitate the training of new assessors, assessments can be observed with the agreement of the trainee and the assessor conducting the assessment. Observers must not interfere with or disrupt the assessment and cannot contest the outcome of the assessment determined by the assessor. The presence of an observer should not have a detrimental impact on the trainee’s performance in the assessment.
2.2.3 Observed assessment outcomes
- Trainees need to meet the criteria outlined in the mark form for the observed assessment being undertaken to achieve a pass outcome. The marking criteria reflect the expectations for the trainee’s stage of practice; there is a mark form for rotation modules, and a mark form for specialty and core modules.
- For all observed assessments undertaken, a mark form should be completed, and the outcome logged in the e-portfolio regardless of whether the outcome is a pass or a fail.
- Where the outcome of an observed assessment is fail, feedback provided by assessors can be used to improve practice and guide a plan for reattempts.