OSCEs by Cees van der Vleuten
I was attending the annual conference of the European Board of Medical Examiners (EBMA) in Łódź in Poland. There was a plenary on OSCEs by Richard Fuller. He gave a masterful overview and was wondering about the future of OSCEs. His central tenet was that if OSCEs do not move along with modern assessment conceptions, it will not survive.
He also speculated about future OSCEs using Star War technologies such as holograms and Virtual Reality. My own love for the OSCE has tempered over time. I started enthusiastically. My own PhD was about OSCEs. It resulted in a review paper with Dave Swanson which has been heavily cited.1 Many years later we updated this review.2 Both papers had a strong psychometric perspective. The biggest message was that sampling is very important and that many stations are needed. Most OSCEs in educational practice are not that reliable because an insufficient number of stations is used. Given the test length required, the OSCE is actually no more reliable than any other measure, including the old fashioned more subjective clinical methods.3 However, my OSCE appetite was not so much tempered by psychometric arguments but by educational ones.
We introduced OSCEs very early in our curriculum at the end of the seventies. We assessed fragmented skills (examination of the abdomen, taking a history, etc.) or part-tasks using elaborate checklists. Learners therefore learned skills in isolation. Moreover, learners memorized the checklists leading to rather trivial behavior without deeper understanding. Naturally we moved to more whole-task stations and more holistic judgments. But still, the OSCE remains a stressful simulated setting. We did a study once where the residents in a postgraduate setting said that they are prepared to jump through the hoop of an OSCE, but we should realize that it had little to do with how they behave in clinical practice.4
I am afraid that the OSCE is much about how to do OSCEs and not a guarantee how learners perform in real practice. I therefore cherish the move towards more authentic work-based assessment, naturally realizing that there are issues with that as well. In our own curriculum we have significantly reduced the number of OSCEs. In our bachelor program (the first 3 years of medical training) we introduced a programmatic model of skills assessment. Whenever students feel ready for a certain skill, they can book a teacher in the Skills-lab in a feedback session on the skill. Learners have skills mentors who follow the learner’s development on clinical skills. The only OSCE left is at the end of the third year, just before they enter the clinical clerkships. It is kind of a readiness test: are you able to advance to the real clinical setting? In the clinical master phase of the program we have no more OSCEs. Instead we have an elaborate programmatic system of work-based assessments and a portfolio. In my travels I often note that the OSCE is seen as the holy grail of assessment. As may be clear from the above, I do not share that view.
By the way, next year the EBMA conference will be held in connection with the AMEE conference (4-6 September, 2020) and Ronald Harden, the founder of the OSCE, will give his view on 50 years of OSCE use in a plenary. I thoroughly look forward to that.
1Van der Vleuten, C. P., & Swanson, D. B. (1990). Assessment of clinical skills with standardized patients: state of the art. Teaching and Learning in Medicine, 2(2), 58-76.
2Swanson, D. B., & van der Vleuten, C. P. (2013). Assessment of clinical skills with standardized patients: state of the art revisited. Teaching and Learning in Medicine, 25(sup1), S17-S25.
3Van der Vleuten, C. P., & Schuwirth, L. W. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39(3), 309-317.
4Van den Eertwegh, V., van Dalen, J., van Dulmen, S., van der Vleuten, C., & Scherpbier, A. (2014). Residents’ perceived barriers to communication skills learning: comparing two medical working contexts in postgraduate training. Patient Education and Counseling, 95(1), 91-97.