Clinical audit made easy

Are clinical audits a mystery to you?  This is your chance to learn the ropes, as Farah Janmohamed explains

According to the National Institute for Clinical Excellence, a clinical audit is “A quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change.”¹

Clinical audits measure the clinical practice of health professionals, such as doctors or nurses, and establishments, such as hospitals or general practices, against standards. Standards define acceptable levels of patient care and healthcare delivery, and seek to ensure that the best care is delivered. They play a fundamental role in audits, and you cannot have one without the other.

Every medical student should grasp the opportunity to carry out or become involved in an audit. Special study modules or firm placements allow you to study topics that interest you, and can be the perfect time to become involved.

As final year medical students, some colleagues and I set up our own audit during our special study modules’ block, but unfortunately we could not find a concise and easy to follow guide to help us. After encountering numerous avoidable problems, we have learnt a great deal about the practicalities of designing and carrying out an audit, which we would like to share with you.
 

The difference between an audit and research?

“Research is concerned with discovering the right thing to do; audit with ensuring that it is done right.”²

You are likely to be asked at some stage, often at interviews, what the difference between an audit and research is. In simple terms, audits look at whether or not the right thing is being done, whereas research potentially discovers the right thing to do. The table summarises the main differences.

Figure 1 summarises the main stages of an audit, which we will be discussing.

Fig 1 Stages of an audit

What can I audit?

The topic you choose should be considered to be clinically important, either locally within your hospital, or nationally.

Seeking out existing national or local standard guidelines, for example, from the National Institute for Clinical Excellence or the appropriate royal college, could be useful in finding a starting point. Perhaps you could identify areas where problems have already been recognised, for example, from patient complaints or adverse incidents. Alternatively, an audit that could possibly result in better resource management is likely to prove popular. It is essential, however, to recognise your limitations early on, especially in terms of time, resources, practicalities, and funding.

Doing a literature review, and using search engines such as Medline, will consolidate your background and fill any gaps in your knowledge.

Who can help me with my audit?

You can always carry out the audit with a fellow medical student, as two brains are often better than one. It is worth discussing your ideas at the planning stage with appropriate team members, such as research registrars (who are generally very keen to help), and other professionals. The support of colleagues (for example, consultants) who have the authority and commitment to implement any changes will make life much easier. Taking your ideas to the clinical audit team in your hospital or trust will determine their practicality and gain consensus.

How long does it take to carry out an audit?

Audits can vary in the length of time taken, depending on your data sources, collection methods, and the numbers required for a representative sample. Before starting any audit, you need to determine how much time you have available to plan, perform, analyse, and possibly re-audit. You may well be surprised at how long it takes to set it up, especially if there are a large number of people involved and little consensus about its direction. The most important thing is to remain realistic about how much time you have to see the audit through—you may only be able to be involved in parts of the audit, and need assistance for its completion. Remember, the results of a lengthy audit will become irrelevant if clinical practice changes by the time of audit completion.

How do I get started?

Once you have chosen your topic, the next step is to set your objectives, that is, what you are trying to achieve. Box 1 should provide some help when putting your objectives together.

You should then think about identifying the standards that will underpin the audit. They can be based upon national guidelines and protocols, research findings from your literature review, clinical experience, and current practice. You must make sure, however, that there is agreement within your department that the standards represent the best and most up to date practice. Standards should be SMART (box 2).

Do I need  ethical approval?

Although clinical audits do not require ethical approval, you must conduct them within an ethical framework. Make sure that the clinical audit team in your hospital have given the go ahead before you start.

What information do I need?

By this point, the information you will need to collect should be clear, for example, the waiting time of patients in accident and emergency. Remember, however, to collect only what is necessary. Collecting extra data will breach the Data Protection Act of 1998, and also be a waste of your precious time.
 

How will I collect the data?

Data can be collected either retrospectively or prospectively. Retrospective collection is usually faster, but be aware of missing documentation. Prospective collection, because it is ongoing, can be time consuming.

Data can be collected in many ways, including the use of proformas (designed data collection sheet), interviews, case notes, and patient administration systems, which process the administrative information related to patients’ contacts with hospitals.

What needs to be included in a proforma?

When designing a proforma, the title and date of the audit should be clear, as well as the contact details of the audit leader. The form should be easy to complete, with explicit instructions for completion and details of where to return the form. You should code the patients to maintain anonymity. An easy way to do this is by numbering your forms from one to 50, and keep a record listing these numbers against hospital numbers. If identification is necessary for any reason (for example, when doing a prospective study that follows the patient’s pathway of care), only a hospital identification number should be recorded. You must make sure that all data is stored securely and in accordance with the Data Protection Act. A clinician’s anonymity must always be maintained unless you have their permission when presenting the results.

Should I pilot my data collection method?

Before starting to collect the data, piloting your tool (for example, your proforma) first will help you to identify any problems you have previously overlooked, especially if the data is going to be collected by a number of individuals. Piloting is done by trying out your chosen method on a small sample, and it will help you to identify whether the collection method is appropriate and usable (for example, the form might be difficult to complete). The aim of piloting is also to ensure that your method enables you to meet the objectives and standards you have set. Piloting may take a bit of extra time, but in the long run it will avoid a lot of headaches, as well as ensuring that the data you collect is both reliable and valid.

How large should a sample be?

There is no right or wrong sample size, but it should be representative of the study population. It is worth noting that postal questionnaires tend to have a low response rate, and so require a larger sample size.

How should I select my sample?

You need to determine who you will and will not include, which will form your inclusion and exclusion criteria, for which you should have some rationale. You should be clear about the time frame from which your samples will be drawn, for example, patients seen in the past, or the next, six months. Don’t forget, you cannot randomly select your cases in an audit.

Who will analyse the data?

If you are a statistical whiz then you may be able to do this yourself, often with the help of a computer program such as StatsDirect or SPSS. Alternatively, you can get help from a statistician, or an audit department if you have produced a proforma.

How can I analyse my data?

The aim of data analysis is to compare your findings with your standards and identify any trends. Calculating the numbers and percentages of cases that have and have not met the standards will help you see whether or not they have sufficiently been met. If not, try to identify the reasons why, so that you can address these later when making your recommendations for changes.

There are several different approaches to quantitative data analysis. Descriptive statistics are used when you want to illustrate numerical data; frequencies, means, medians, ranges, and standard deviations are examples. This data is often best displayed graphically, for instance, in the form of bar graphs or pie charts. Statistical tests can be used for comparisons, for example, with before and after data, or to determine whether or not the results are statistically significant.

It is worth noting that qualitative data, collected from open questions on your proforma, can be tricky to analyse and you are likely to need help. Hence, most audits only collect quantitative data.

What do I do with my findings?

Once the data has been analysed, you should present your findings to fellow colleagues and other key staff members before considering writing a report. This will provide you with an opportunity to generate discussion about the results and identify whether any new changes may be implemented or if existing guidelines and standards should be updated. Ideally, the presentation should be formal and you should include:

• Title slide
• Background
• Aims and objectives
• Standards
• Methods
• Results
• Conclusions
• Points for discussion
• Recommendations for practice improvement
• Proposed action plan
• Summary
• References (often obtained from literature used to identify standards).

After the presentation, you might want to think about writing an audit report. The discussion from the presentation will be important in identifying factors such as limitations and areas for improvement. The report should clearly outline each stage in order to allow replication or re-audit at a later date. It is worth remembering that your audit might be repeated again by others, and the report acts as a record of the study. The layout of the report is similar to that of the presentation, but with a lot more detail. You should, however, include at the start of the report, a contents page and an abstract that summarises the main body of the report. You may also need to include appendices at the end—this is where documentation such as your proforma would be placed. We highly recommend thinking about the contents of your report from the start of the audit; essentially, there is no reason why you cannot plan beforehand everything up until the results section. Copies of the report should be given to all of those involved, or possibly implicated, as this is an efficient way of feeding back your findings.

If changes need to be made, how do I put them into action?

A consensus is essential in bringing about any changes, and therefore your audit recommendations should be valid and reliable. Don’t try and implement changes if they are not improvements. Figure 2 summarises the main stages in making the necessary changes.

Fig 2 Stages involved in implementing changes³

Why is there a need to re-audit?

Re-auditing is important to confirm that any necessary changes have been made, that these have had the desired effect, and that standards continue to be met. The process is the same as the initial audit cycle, except certain stages (such as doing a literature review) may not need to be repeated. It is not uncommon to have to repeat the process several times until you are satisfied that the improvements to clinical practice are being upheld.

Can I publish my audit?

Audits are sometimes published, particularly if the results are generalisable and if others may also accrue the benefits. Editors are usually more interested in the methodology and lessons learnt in setting up the audit, rather than the results themselves. Carrying out a re-audit, which will demonstrate how practice has improved, will increase the likelihood of publication.

photos.com
Is your clinical practice up to standard?

A final note

Although audits can seem daunting and challenging, it is definitely worthwhile getting involved in one if the prospect arises. Not only are they an integral part of routine clinical practice, which you will inevitably be involved in at some stage, they look great on your curriculum vitae and job application forms. Getting a publication in a peer reviewed journal or presenting a poster are added bonuses. Audits also provide the chance to build on your teamworking skills, not to mention being a great networking opportunity. We hope this guide will help reduce some of the headaches associated with setting up your own audit.

We thank A Naftalin, R Chenoy, and Professor O Djahanbakhch, Academic Department of Obstetrics and Gynaecology, Newham University Hospital (Barts and The London, Queen Mary’s School of Medicine and Dentistry).


Farah Janmohamed, final year medical student, Barts and the London Medical and Dental School
Email: farah3006@hotmail.com
Laura J L Halpin, Barts and the London Medical and Dental School
Email: halpin_laura@hotmail.com
Sanjay Patwardhan, Academic Department, Newham University Hospital, Barts & The London Medical & Dental School
Email: sanjay.patwardhan@newhamhealth.nhs.uk


studentBMJ 2006;14:1-44 January ISSN 0966-6494

1. National Institute for Clinical Excellence. Principles for best practice in clinical audit. London: NICE, 2002.
2. Smith R. Audit and research. BMJ 1992;305:905-6.
3. www.rcpsych.ac.uk/publications/gaskell/samplechaps/clinauditChap2.pdf (accessed 2 Dec 2005).