The Journal Article was ‘Workplace-Based Assessments in Postgraduate Medical Education: A Hermeneutic Approach’. The paper investigated workplace-based assessments for medical postgraduate students. It identified twelve main lessons that can be used to improve such assessments. These were; (1) using formative and summative assessments, (2) ensuring that assessors are all trained in workplace-based assessments, (3) ensuring students receive high-quality narrative feedback, (4) using scales that are meaningful to stakeholders, (5) ensuring communication of the scales used and how they’re assessed, (6) ensuring that benchmarking data is available, (7) clearly articulate to what extent the purpose of the assessment is formative or summative, (8) if used for a summative purpose, the assessment should be collated longitudinally, (9) regularly assessing the culture and environment created by the assessment, as more competitive cultures lead to more cheating, (10) enforcing consistency and minimum standards across the board, but ensuring there is flexibility to tailor to individuals needs, (11) measures need to be taken to ensure the students’ performance is as authentic as possible, and (12) assess students in a variety of contexts and with different assessors.
The group began by discussing the use of the word ‘stakeholders’ in the study, it was mentioned how the stakeholders were explicitly not students, nor academics, but there was little mention in the paper of who the stakeholders actually were. There was also a continuing discussion about how students and academics should count as stakeholders – anyone affected by the assessment has a stake in it. Attendee’s then discussed the fact that assessors tend to (often unconsciously) assess others based on their own experiences, if they find a certain style of exam hard, they are likely to grade others harsher in it, therefore there is a need to ensure that the people assessing the workplace-based assessments are trained and qualified enough to assess as fairly as possible. The group linked back to a previous masterclass in this discussion, talking about how there is no such thing as a value-neutral assessment, and how every exam or assessment has unconscious bias woven into it. The group also discussed the concept of formative assessments, and how they were valued by both academics and students. With students seeming to value the narrative feedback that they often bring, without the heavy consequence of failing. There was talk surrounding how students taking the assessments learn from their academics example, which means that designing authentic assessments is very important – because if a student is just given a multiple choice exam, or a computer screen, and no practical assessment, they will take on that the theory is more important than how you, for example, treat patients.
When discussing the paper as a whole, the group found it to be overly verbose and time-consuming to read. The paper appeared to repeat itself, without bringing in any new information. The group did find that the box at the end of the paper which summarised the twelve lessons was very helpful, and probably sufficed to explain them all. There was some discussion surrounding whether the group had read the paper as it intended to be read, one academic commented that if they had read the paper as we teach students to (reading the abstract first, conclusion second and then the rest of the paper) then the paper may have made a lot more sense the first time they had read it. There was also a talk about how most of the academics were coming from healthcare or scientific backgrounds, with have a heavy focus on conclusions and being concise, versus other academic fields where more of the focus may be on the paper itself. This could be a potential reason why the group found the paper to be useful, but very long-winded and verbose.
The take-home messages from the participants were:
- What workplace-based assessments are like from the perspective of students and educators, and how to question preconceptions around that
- The way to read the paper is to read the abstract, then conclusion and then ultimately decide if you want to read this paper – reading through from the start to the end was not helpful here
- Thinking about whether the 12 outcome points identified could be used in a different setting (for example, the University) and be more academic focused
Continue the conversation through our Yammer Pedagogy Journal Club.
S Prentice, J Benson, E Kirkpatrick and L Schuwirth (2020) Workplace-based assessments in postgraduate medical education: A hermeneutic review, 54: 981– 992, DOI: 10.3109/0142159X.2013.800636