Valerie Wass argues that the OSCE is the fairest assessment method The objective structured clinical examination (OSCE) is becoming the favoured method of clinical competence testing in undergraduate and postgraduate examinations. The examination, however, requires a high level of funding - more examiners, patients or role players, and equipment are necessary. The traditional long and short case examinations were more fun for the examiner, who could interact freely with the candidate, and they were, perhaps, more true to life. The new examinations may feel artificial and rather reductionist.

Why should we change?

So why the change? The answer hinges on reliability. The stakes are high in clinical examinations - key decisions about whether the candidate qualifies as a doctor are being made. The pass or fail decision has to be consistent. There has to be a high degree of certainty that the candidates would be the same side of the pass or fail line if they repeated the examination the next day.

A key feature of clinical testing is that medical students and doctors perform inconsistently across tasks.

This is hardly surprising given the varied experiences through which we learn. Clinical teaching relies on patient availability, and we all learn from a range of different ward environments and specialty mixes. Some are well taught and others less well.

Unless a clinical test covers a wide range of different clinical contexts, it lacks reliability. Chance has the upper hand. We know that for an undergraduate examination a minimum of 20 to 22 different stations are necessary. This is in contrast to the old examinations of four to five cases. The people who argue for the old traditional method are on quicksand. Overall the OSCE, although not ideal, is the fairest assssment method. An examination cannot be valid if it is not reliable.

Miller's triangle, which stages clinical competence, is a helpful model (see figure). Tests appropriate to each level have been added to demonstrate the role of the OSCE. The ideal level of clinical testing would be the "does" - that is, a reliable test of student performance on the wards. This has to be the focus of assessment research in the 21st century. The OSCE tests at the level of "shows how." This is undoubtedly fairer than an unreliable sign up system for clinical attendance. It can also be argued that unobserved case presentations or the traditional long case assess only at the "knows how" level.

An OSCE tests more than knowledge. Correlation of student performance across different types of test shows this. Of course, some knowledge is needed but you cannot study for an OSCE in the library. The knowledge gained is from experience on the wards, and proficiency in practical and clinical skills comes only from practice. It is not enough to learn the checklist. The more the skill is performed, the more automatic and fluent the performance becomes. Students scoring highly on knowledge tests cannot expect to score highly in OSCEs. There is a correlation but not a strong one.

Most medical schools set their OSCEs at minimal competence levels. The test is not designed to identify brilliance. The pass rate is set at a predetermined minimum safety level below which the candidate is felt to be "unsafe" to work with patients.

Many students set very high standards. OSCE results can be disappointing. Very few students pass all the stations. For example, in a recent OSCE which I ran for 330 students, only 12 (less than four per cent) passed all the stations. Some of those who failed one or two stations did better overall than these 12. Do not be disappointed. Remember: all doctors are good at some things and less good at others.

The box sets out the key issues in passing the OSCE test.