Corticosteroids in acute respiratory distress syndrome: meta-analysis

Acute respiratory distress syndrome (ARDS) is a serious illness with an appreciable mortality rate, which occurs in patients who are ill from some other cause, such as sepsis. It is characterised by stiff lungs, diffuse bilateral pulmonary infiltrate, hypoxaemia, and the absence of cardiogenic pulmonary oedema. Clinical features consist of tachypnoea, increasing hypoxaemia, and laboured breathing. Chest x ray images shows diffuse bilateral shadowing, which may eventually progress to complete whiteout.

The introduction to this paper is admirably brief and states that the use of steroids in this illness is controversial. The aims of this study are to review the evidence for using steroids as a preventive measure and as a therapeutic measure. Previous overviews of the evidence have been contradictory, so this might seem a good reason to do a further review, using more rigorous methodology.

What did the authors do?

This study design is a meta-analysis, a statistical synthesis of the numerical results of several trials that have all tried to tackle the same question.^w1 Meta-analysis is a logical development of a systematic review; the latter is simply a pulling together of the evidence about a topic in an attempt to produce a consensus on the outcome measure. Meta-analyses quantify the overall effect of an intervention in a single statistic, derived by mathematical manipulation of the results from individual trials.

The credibility of systematic reviews and meta-analyses depends on the rigorous gathering of all the evidence available on the subject, using different search methods on databases; writing to authors; searching for unpublished trials, for example, in drug company records; and so on. In this trial the search was done electronically and by visual search of reference lists in articles, and the only evidence included was from trials and not from observational studies. A rigorous definition of the disease ARDS was used, according to 1994 US-Europe consensus, and the studies were assessed for quality using a 10 point scoring system.

In the hierarchy of evidence, systematic reviews and meta-analyses rank as the best quality and so are of great importance in evidence based medicine. However, it is not easy to do a good systematic review or meta-analysis because finding all the evidence on a single topic is sometimes convoluted and tedious. Occasionally it is necessary to write to authors to get further results that may not have been published in the final version of the paper. Other papers may never have been published even though they contain important data: only persistence on the part of the researcher will retrieve them. And some trials are never published because their results may embarrass the sponsor.

Heterogeneity of trial data, wide disparity in the results, will also cause problems in the analysis. Rough estimates of statistical heterogeneity can be gauged by looking at the results and seeing how much the confidence intervals overlap.

Statistical analysis

This merits a special mention because the authors used Bayesian models, as opposed to more conventional “frequentist” statistics. Frequentist means that the probability of an event is based on the long term frequency of that event in a series of identical experiments. This concept has shortcomings, importantly in the interpretation of P values and confidence intervals, and Bayesian statistics goes some way to resolving these problems.^{w2 w3}

The Bayesian approach starts with a prior belief about the likely values of variables, which are entered into a statistical model, and then observed data are used to modify these likely values. The prior belief concerning the value of the variables is often expressed as a range (the prior distribution) and is simply a quantification of the current state of knowledge about that variable.

In this paper, the prior distribution is classified as “disparate,” meaning that it includes a wide variety of starting values because of the scarcity of evidence about the true value. The range is narrowed using computer programs that run repeated iterations of the model.

The outcome (the posterior distribution) is the probability distribution of the parameters once all the data are entered into the model.^w4 In this study the posterior distribution is presented for the primary outcome measure—deaths of patients who developed ARDS—and for several secondary outcomes, such as the mean difference in the number of ventilator-free days.

These results are shown in two ways. An odds ratio^w4 with 95% credible intervals (95% probability that the true value lies between these values) and the probability of the odds ratio being 1 or more were calculated for death. In this study probabilities are expressed as a percentage, not the more usual frequentist convention as a fraction between 0 and 1.

What did the authors find?

The number of articles included in the analysis is small, at just nine, of which four evaluated preventive therapy and five assessed the role of steroids after the onset of ARDS. The total number of patients taking steroids in all of the studies was about 560. These are relatively small numbers in terms of meta-analyses, and the division into two subgroups (preventive and therapeutic) further reduces the power of the data.

Fig 1 and fig 2 (figs 2 and 4 in the long version of the paper on bmj.com) give a summary of the results, with reference to deaths of ARDS patients from preventive and therapeutic steroids compared with deaths from placebo. This type of figure is known as a Forest plot. Each row represents one study, with the odds ratio shown as a number with a point estimate and a line for the range of the 95% credible interval. The size of point represents the size of the study in participant numbers. The bottom row shows the summary odds ratio for all the studies, a diamond with its mid-point at the odds ratio and edges that represent the 95% credible interval.

Fig 1 Forest plot to show effect of use of preventive steroids on subsequent mortality of those who developed ARDS

Fig 2 Forest plot to show effect of use of therapeutic steroids on subsequent mortality of patients with ARDS

In both figures1 and 2 the diamond crosses the line of 1, and so we cannot rule out the possibility that steroids have no effect on an outcome of death in either circumstance. Note that in fig 1 the estimate of heterogeneity among studies is 0.97 (measured as the standard deviation between the studies), indicating considerable differences between the individual studies. The heterogeneity in fig 2 is smaller (0.53), indicating more agreement among the studies.

Looking at the probability values in the abstract (remembering that this refers to the probability of the odds ratio being at least 1, where 50% indicates a null effect), the probability of an association between steroid treatment and the development of ARDS was 86.6%. This shows that there may be an adverse effect from giving steroids as preventive therapy.

Similarly, the probability of more deaths from giving preventive steroids was 72.8%. Conversely using steroids in established ARDS seems to decrease deaths (probability of 6.8%). Of course, as mentioned above, the Forest plots in fig 1 and fig 2 show that a null effect is still a possibility in both situations.

Also, therapeutic steroids increased the number of days without mechanical ventilation (mean difference 4.05 days, 95% credible interval 0.22 to 8.71), and there was no increase in secondary infection, which you might anticipate as a side effect of this treatment.

What does this mean?

The evidence presented in this meta-analysis is not conclusive because it contains too few trials with too few patients to answer a number of questions—for example, what is the optimum dose and duration of treatment? Results for the use of preventive steroid therapy are rather heterogeneous, but the Bayesian probability that harm ensues, in terms of patients subsequently getting ARDS and in deaths, will probably put most doctors off this as an option. Conversely, the evidence for the use of steroids to treat established ARDS is more encouraging but might not convince everyone.

Whatever the type of statistical analysis, if the primary data are inadequate in their number and quality then it is impossible to produce adequate and convincing answers. Bayesian models deal with uncertainty better than frequentist statistics because they allow computation of a probability of harm or benefit even if the confidence intervals cross the line of null effect. However, this is at the expense of the fact that it requires an estimate of a prior value for the measure, and it is not always clear what that should be.

Perhaps the most compelling conclusion from this meta-analysis is that more research is needed. This age old aphorism accompanies the conclusions of many clinical trials, so we look to meta-analyses to produce definitive answers. As such, this study is unusual with regard to its uncertainty.

Competing interests: None declared.

Provenance and peer review: Commissioned; not externally peer reviewed.

See the editorial that accompanied the paper (BMJ 2008;336:969-70; www.bmj.com/cgi/content/full/336/7651/969).

1. Greenhalgh T. How to read a paper. 3rd edition, 2006 BMJ books/Blackwell publishing Oxford.
2. http://www.bbc.co.uk/dna/h2g2/A801695 accessed 14/05/08.
3. Burton PR, Gurrin LC, Campbell MJ. Clinical significance not statistical significance: a simple Bayesian alternative to P values. J Epidemiol Community Health 1998; 52: 318-323.
4. Kirkwood BR and Sterne JAC. Medical Statistics. 2nd edition, 2003. Blackwell publishing, Oxford.

   

 Printable version    Download PDF   E-mail this to a friend   Respond to this article