Should candidates be given their marked up papers back after examinations - the case against
Professor A P Weetman explains the reasons why papers may not be returned
I have a lot of sympathy for the views expressed in Varma's article. There are, however, a number of points that, I hope, can be addressed to provide reassurance to student readers. Before doing so it is important to address the question of why papers are not returned. This is usually a matter of university policy in general.
There are three main reasons for not returning scripts. The first is mentioned in the article, namely that questions are often reused, particularly if they have been proved to be good discriminators. The second reason is the sheer administrative burden entailed for any university in distributing scripts back out to candidates, most of whom will have passed and will have no desire to see their scripts. The third, but most important, reason is that the scripts are part of the university's quality assurance mechanism, and both external and internal assessors can use stored scripts dating back several years to test a university's quality assurance programmes. For example, the Quality Assurance Agency, which is currently visiting medical schools on behalf of the Higher Education Funding Council for England, looks at scripts dating back three years. The counterargument to this point may be that students could return their scripts once they have seen them, but, in my experience, this never works.
Individual feedback may help
It is a matter of local policy what marks are fed back to students, but it is certainly my practice to give a detailed feedback on performance, on an individual basis, to any student who has failed his/her final examination. This includes, if the student wishes, a report on the marks for each section of a paper. Thus, although it may be university policy not to return scripts, schools may well have robust and appropriate feedback mechanisms that will prevent some of the problems mentioned by Varma.
Of course, there is also a move to much greater formative assessment within medical courses, which provides exactly the sort of feedback that is necessary for students to gauge their own performance. Examination scripts cannot provide this depth of feedback as examiners' comments are usually not included on the scripts. The reason of this is that, with double marking, each examiner has to see the script as originally submitted by the student rather than with the bias of another examiner's comments on the script. My own experience from double marking of examination scripts is that there is rarely a serious discrepancy between markers, provided model answers are used, and a rounding up process for two discrepant marks will favour the student. External examiners provide another line of protection as they, in general, focus on those scripts which lie at the extremes of the marking range.
Formative v summative assessment
I will now deal with some of the points raised by the article which need further amplification. There is no doubt that formative assessment, as described above, is an essential part of the new curriculum proposals suggested by the General Medical Council. Formative assessment should, however, not be confused with summative assessment, which is a definitive exam that determines whether or not a student progresses to the next phase of their course. Particularly in medicine, where the examination confers the right to practice, summative assessment has a different role to formative assessment.
Anonymous marking is a robust system when properly practised. The fact that some students may not be able to remember their candidate numbers is not really a viable argument for stopping anonymous marking. I sometimes wonder whether being able to complete the simple instructions should not be part of the examination itself - doctors have a lot of forms to fill in.
Although no one can guarantee complete accuracy in any system devised by human beings, there are a number of fail safe mechanisms built into examination marking. In particular, undergraduate medical examinations tend to pass most candidates, and therefore it is comparatively simple to check the marks of those that fail. The marks are checked at several examiners' meetings. Marks for individual components of the examinations are now put on to spreadsheets, and this is the only time that mistakes can be made, although careful checking mechanisms are in place.
Calculation of total scores is now carried out using formulae in spreadsheets, and the profile of marks is analysed. Scripts of borderline candidates are often moderated and available to examiners for oral (viva voce) exams. I am more than happy to review individual component marks with candidates, including those who believe that a mistake has been made, and I am sure no school would refuse to check marks if a student is concerned about accuracy. Formal appeals mechanisms exist which can be used to examine any suspected procedural error.
OSCEs v clinical short cases
Varma does seem to understand how an objective structured clinical exam (OSCE) runs. During an OSCE a marker (sometimes a pair of markers) sees all students going through one particular station, and most OSCEs are 10-20 stations (not 50). The OSCE is a fairer style of examination than the clinical short cases which it has largely replaced largely.1-4 All students see the same material, and there are no vagaries in the marking as each component of a response has a prearranged mark. The marker then simply has to make a decision whether the candidate has completed that part of the examination correctly, which usually entails awarding one of several possible marks for that component. The marks are given as the candidate proceeds through the station rather than at the end.
Varma's misapprehensions also extend to his statements about clinical examinations and vivas. All examiners keep a record of the student's performance in these examinations, and if it becomes obvious that a student is failing, very detailed notes are kept. All students who fail these parts of the examination, in my experience, are discussed individually with the panel of examiners present, which means that examiners have to provide a defence of the mark that they give. It is important to emphasise that at least one external examiner will be present and, therefore, the marking process is open to external scrutiny. Finally, in the medical schools I have examined at, there is a fail safe mechanism built in to finals examinations whereby the student is given a second chance with either a pass/fail viva or repeat short cases. In these circumstances the candidate is seen by a fresh external examiner and an internal examiner, and once again copious notes will be made during this process. Therefore, any student who wishes to have an explanation of his or her performance can obtain it by asking whoever is in charge of the examination to go back through the notes. In my experience, very few students are surprised at the outcome of the examination, and I have yet to come across anyone who has challenged the marks when these are explained.
Conclusion
To summarise, examinations are much fairer than Varma believes. It is certainly our practice in Sheffield to provide an explanation of the marks achieved in a summative examination to any student who requests it but examination scripts are retained as part of internal and external quality monitoring. The retained scripts are the university's proof that mistakes, such as those Varma is worried about, are not made. Summative assessment has a different purpose to formative assessment, and I totally agree with the need to provide abundant forms of formative assessment at appropriate times during the course. I believe that all medical schools should be prepared to disclose the marks for individual components of a summative examination, and I am not aware of any school that has refused a reasonable request to check marks if a student feels there has been a mistake.
A P Weetman, dean, Sheffield University Medical School, Sheffield
studentBMJ 1999;07 December ISSN 0966-6494
- Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13:41-54.
- Newble DI, Swanson DB. Psychometric characteristics of the objective structured clinical examination. Med Educ 1988;22:325-34.
- Roberts J, Norman G. Reliability and learning from the objective structured clinical examination. Med Educ 1990;24:219-23.
- Matsell DG, Wolfish NM, Hsu E. Reliability and validity of the objective structured clinical examination in paediatrics. Med Educ 1991;25:293-9.
