Children's bones and calcium supplements:meta-analysis

Meta-analyses combine the results of separate randomised controlled trials. Elizabeth Loder looks at one that shows parents might have got it wrong when they recommend their children drink milk for strong bones

This month's paper is "Effects of calcium supplementation on bone density in healthy children: meta-analysis of randomised controlled trials" by Winzenberg T and colleagues (BMJ 2006;333:775-80).You can read the paper by going to studentbmj.com and clicking on the link.

Calcium is largely deposited in our bones, where it is important for bone mass and strength. Bone is metabolically active, so calcium is constantly being added and removed. If the amount of calcium removed regularly exceeds the amount added, bones can become thin and weak. The density of bone is an indirect measure of bone strength and can be measured by bone mineral density tests.

Bones naturally become thinner and weaker over time, but how soon that causes problems depends on a person's peak bone mass, which is reached in the late teenage years. The higher the bone mass is to start with, the longer it will take for bones to become so weak and thin that they are at risk of fracture. This means that adequate calcium intake in childhood is important.

The paper describes a meta-analysis to see whether calcium supplementation improved bone mass in healthy children, and also to see if anything else influenced the effects of calcium and whether effects persisted after supplementation was stopped. The project was initially published as a Cochrane review, and now it has been republished in the BMJ.

The term "meta-analysis" refers to statistical methods that combine the quantitative results of separate but similar studies. The standard way to do a meta-analysis is to pose an important, focused question that has not been satisfactorily answered by individual research studies. At this stage of the process it is necessary to precisely define what kinds of studies are appropriate to answer the question. This means deciding on the population and hypothesis to be studied; the intervention and comparison groups to be studied; the outcome measures of interest; and the study designs that will be considered.

The next step is a careful review of the medical literature to find all relevant research studies about the problem. This is usually called a systematic review to emphasise the fact that careful, objective search strategies are used to minimise bias and to maximise the likelihood that all the important articles about the subject will be found. The quality and relevance of retrieved studies is then carefully evaluated. Only studies that meet the criteria determined in the first step of the process are selected for the review.

The data from individual studies is then collected, generally by at least two different researchers who use standardised forms and compare their results as a way to minimise errors. Sometimes the researchers need to derive the data from other information in the paper. For example, some studies report only the percentage and not the actual number of subjects with a certain outcome. In such a case, the authors may need to calculate that number based on other information in the paper.

The quantitative information extracted from the studies is then entered into a database, and statistical methods are used to combine it. A description of statistical methods used to perform meta-analyses is beyond the scope of this article, but further information is widely available. At a minimum, a meta-analysis should present the individual numerical measures and confidence intervals for all of the individual studies, as well as the single measure that is calculated by combining data from the studies. The results must then be interpreted and the clinical implications discussed.

The most common reason for a meta-analysis is that many similar studies have been done with contradictory or unclear results. Sometimes small studies lack the statisticalpower needed to detect an important treatment effect or to give a precise estimate of the effect. Combining their results is cheaper and faster than doing a new, large trial.

In this meta-analysis, the researchers decided to include only studies of healthy children who were randomly assigned to receive either placebo or extra calcium as a supplement or in food for at least three months. They also limited their review to studies that measured "bone outcomes" after at least six months. Children in the studies had to be younger than 18 years old and free of medical conditions that might affect bone metabolism. The researchers included studies that used any of five different methods of assessing bone density.

To find these studies, the authors did a computer search of multiple databases and hand searched the references of articles they located as a way of making sure they did not miss relevant articles. Two researchers examined each retrieved article to see if it met the predetermined criteria for inclusion in the review. They then rated the quality of selected studies, based on such things as whether the report of the study provided an adequate description of blinding, whether the randomisation and allocation procedures used were adequate, and so on.

The researchers entered the numerical information from the studies into a database. For this meta-analysis, the authors converted the outcome measures from the original papers to standardised mean differences using special software. They also evaluated how similar the original studies were, to decide on the best statistical model to use to analyse the data. Finally, the authors compiled graphs showing the individual effect sizes of the trials that contributed data to the endpoints examined in the meta-analysis.

The researchers found 19 trials that met their predetermined inclusion criteria, in which a total of 1367 children were assigned to calcium supplementation and 1426 to placebo. Six different methods of calcium supplementation had been used, with doses ranging from a low of 300 mg to a high of 1200 mg of calcium a day. The quality of the methods of the included studies was not especially high-bias was rated as moderate or high in 17. Not all studies provided useable data for every endpoint examined in the meta-analysis.

The combined results of the studies showed that calcium supplementation did not have a big effect on bone mineral density in the healthy children who were studied. Variables such as sex, physical activity level, and pubertal stage did not change this result, and the authors conclude that calcium supplementation in healthy children is unlikely to offer an important public health benefit.

Meta-analysis is a powerful method, but it can be misused. "A meta-analysis is only as good as the studies that comprise it, and as good as the many decisions that were made when designing and performing the meta-analysis..."¹ Common criticisms of meta-analysis are that it combines studies of varying quality and design-"apples and oranges"-and that it may produce a statistically significant result that is of little clinical importance.

An important strength of this meta-analysis is that the authors explain their decisions and procedures in detail. This makes it possible for readers to evaluate its methods and results based on their own clinical knowledge and common sense. A reader can decide whether it makes sense to combine the results of studies using different doses and types of calcium supplementation and evaluate the soundness of the authors' reasons for doing so. This transparency is an important safeguard against a potential pitfall in meta-analysis, which is that despite their statistical expertise, researchers may not be experienced clinicians or experts in the subject.

The authors of this meta-analysis point out the main limitation of their analysis, which is that it did not evaluate fracture, the medical outcome that is really of clinical interest. Instead, it evaluated bone mineral density, which is a surrogate, or stand-in, measure that is highly correlated with fracture risk. The best way to study whether calcium supplements are worthwhile would be a blinded, randomised study of long term bone fractures in children who get calcium supplements compared with similar children who get placebo, but this ideal type of study is impractical. A systematic review or meta-analysis of shorter, smaller studies is probably the next best way to answer the question.

When this paper was published in the BMJ, a reader, Robert Heaney from Nebraska, sent a rapid response about it to bmj.com pointing out that because this meta-analysis looked at calcium supplementation in healthy children with a normal calcium intake, its results do not apply to children whose calcium intake is inadequate. He worried that "it is likely that the message for the average reader would be that calcium intake in children is not important. That would be not only wrong, but potentially dangerous, as well."² This is an important point-the results of this meta-analysis apply only to the studied group. Another important question to study might have been the response to calcium supplementation in children whose calcium intake is inadequate. The same methodical, orderly approach used in meta-analysis can also be applied to appraising its results. The Critical Appraisal Skills Programme has drawn up a list of ten questions that readers can use to evaluate a meta-analysis (box).²

1. Hamer RM, Simpson PM. SAS tools for meta-analysis. Cary, NC: SAS, http://www2.sas.com/proceedings/sugi27/p250-27.pdf (accessed 18 Oct2006).
2. Winzenberg T et al. Effects of calcium supplementation on bone density in healthy children: meta-analysis of randomised controlled trials. BMJ 2006 http://bmj.com/cgi/eletters/333/7572/775#142646