Getting through OSCEs
Valerie Wass argues that the OSCE is the fairest assessment method The objective structured clinical examination (OSCE) is becoming the favoured method of clinical competence testing in undergraduate and postgraduate examinations. The examination, however, requires a high level of funding - more examiners, patients or role players, and equipment are necessary. The traditional long and short case examinations were more fun for the examiner, who could interact freely with the candidate, and they were, perhaps, more true to life. The new examinations may feel artificial and rather reductionist.
Key issues
Study the curriculum
Look carefully at the curriculum and course books. Well planned OSCEs aim to cover the course content as widely as possible. Make sure you know what is being assessed
Understand the skills
Make sure you understand the criteria for the skill. Work these out yourself rather than learn a checklist. Get a good demonstration of the skill to model on
Practise the skills with as many patients as possible
Observe each other. Revise together. Observe and give feedback. It can be fun
Remain confident
You need the right mentality. You will undoubtedfly flunk some stations. Forget it immediately and move confidently on to the next
Why should we change?
So why the change? The answer hinges on reliability. The stakes are high in
clinical examinations - key decisions about whether the candidate qualifies as a doctor are being made. The pass or fail decision has to be consistent. There has to be a high degree of certainty that the candidates would be the same side of the pass
or fail line if they repeated the examination the next day.
A key feature of clinical testing is that medical students and doctors perform inconsistently across tasks.
This is hardly surprising given the varied
experiences through which we learn.
Clinical teaching relies on patient availability, and we all learn from a range of different ward environments and specialty mixes.
Some are well taught and others less well.
What does the OSCE test? Miller's triangle of clinical competence (Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65:563-7)
Unless a clinical test covers a wide range
of different clinical contexts, it lacks reliability. Chance has the upper hand. We
know that for an undergraduate examination a minimum of 20 to 22 different stations are necessary. This is in contrast to the old examinations of four to five cases. The people who argue for the old traditional method are on quicksand. Overall the OSCE, although not ideal, is the fairest
assssment method. An examination cannot be valid if it is not reliable.
Miller's triangle, which stages clinical competence, is a helpful model (see figure).
Tests appropriate to each level have been
added to demonstrate the role of the
OSCE. The ideal level of clinical testing
would be the "does" - that is, a reliable test
of student performance on the wards. This
has to be the focus of assessment research
in the 21st century. The OSCE tests at the
level of "shows how." This is undoubtedly
fairer than an unreliable sign up system for
clinical attendance. It can also be argued
that unobserved case presentations or the
traditional long case assess only at the
"knows how" level.
Tips
- Read instructions carefully. Stick to them.
- If uncertain of the instructions check with the examiner. Clarify any uncertainties - as you start.
- Remember the professional interaction with the patient. Examiners always look for it. Introduce yourself and refer to the patient by name. Don't rush. In communication stations listen to the patients first. They have scripted instructions. Start with an
open question and listen, they will be cueing you the information. Ask about concerns. If asked to give an explanation always elicit what the patient understands first. Never launch straight in.
- Be considerate and gentle when examining patients. Examiners get very upset if patients are not treated with respect. There is often an overall mark (global rating) as well as a checklist where they may mark you down for this.
- Study equipment carefully. As you start a practical skills procedure check the equipment. If you see a sharps box or yellow disposal bag you can be sure there will be a mark for using it.
- Avoid watching the examiners ticking boxes. This can be very misleading.
- Timing. If you are given warning bells - for example, one more minute, use the time to summarise and conclude the interview. There are often marks for this.
- Stay calm. Use rest stations to think forwards not backwards.
An OSCE tests more than knowledge.
Correlation of student performance across
different types of test shows this. Of course,
some knowledge is needed but you cannot
study for an OSCE in the library. The
knowledge gained is from experience on
the wards, and proficiency in practical and
clinical skills comes only from practice. It
is not enough to learn the checklist. The
more the skill is performed, the more automatic and fluent the performance
becomes. Students scoring highly on
knowledge tests cannot expect to score
highly in OSCEs. There is a correlation
but not a strong one.
(ULRIKE PREUSS)
Most medical schools set their OSCEs at
minimal competence levels. The test is not
designed to identify brilliance. The pass rate
is set at a predetermined minimum safety
level below which the candidate is felt to be
"unsafe" to work with patients.
Many students set very high standards.
OSCE results can be disappointing. Very
few students pass all the stations. For
example, in a recent OSCE which I ran for
330 students, only 12 (less than four per
cent) passed all the stations. Some of those
who failed one or two stations did better
overall than these 12. Do not be disappointed. Remember: all doctors are good
at some things and less good at others.
The box sets out the key issues in passing the OSCE test.
Valerie Wass, senior lecturer in general practice, Guy's, King's and St. Thomas's Hospital Medical School
Email: valerie.wass@kcl.ac.uk
studentBMJ 2000;08:347-394 October ISSN 0966-6494
