Should snowboarders wear helmets?

Bias

Sources of bias and the way in which they could influence the results are discussed by the authors themselves if the study is reported well, but you should always make your own judgement. In a case-control design, the major concerns when assessing possible sources of bias are the way in which participants were chosen (selection bias) and the accuracy of data recording the exposure to risk factors (information bias). The later depends on the participants’ memory and willingness to report truthfully (recall bias) and the way in which the data were collected.

The criteria for participants’ inclusion in the study should be well defined and objective. We want our participants to accurately reflect the target population with regard to the study question. If the inclusion criteria are not well defined or are too subjective, researchers will be tempted to, consciously or not, tamper the way in which they assign participants to groups. For example, if the researcher believed that wearing a helmet while sustaining the injury to the neck was harmful, he or she may assign more people who wore helmets to the cases (for neck injuries) and not assign them if they didn’t wear a helmet (although they may have a similar type of injury). This doesn’t have to be conscious manipulation: the researcher may truly believe that the person who wore the helmet was much more seriously injured than the person who did not wear the helmet.

In this study, the assessment of inclusion criteria for both cases and controls was clearly defined and well out of researchers’ reach, as it was based on the records of the ski patrol and hospital staff who did not otherwise participate in the study. Other researchers previously reported that ski patrols accurately report the body regions affected. A classification for severity of injuries used in this study was similar to that used in other studies on the basis of mode of leaving the ski area (for example, by ambulance, with parents). Our researchers also published a separate study, which they reference in this paper, in which they found few differences in consistency of reporting between the postal questionnaire or phone interview and accident report form. These references reassure us that the researchers thought carefully about the way the cases or controls were defined or included in the study, which is very important and one of major sources of bias in a case-control design.

Ideally, we would also want all eligible cases to participate in the study. This is almost always impossible. The response rates (the proportion of people who participated in the study among all those who were invited; here in plural because of several subgroups) were about 70%, which is generally accepted as a decent response rate.

If cases who had not worn a helmet were more confused than controls about whether they had worn a helmet and reported “the exposure” wrongly, that could have lead to underestimation of the protective effect of helmets ie, thinking that they don’t protect as well as they actually do. A concern is also in place about the fact that the authors did not include in injured controls those who fell and hit their heads but did not sustain any injury because they were wearing a helmet. This also leads to underestimation of the protective effects of helmets. In this particular study, this is of lesser concern than overestimating (ie, thinking that they protect better than they actually do), because of the concern that helmets may increase risk of neck injuries. If later was true, we certainly wouldn’t want our overestimated results to wrongly prompt us to advice for helmet wearing—even though let’s say helmets protect from head and face injuries only rarely, and most times increase the risk of the neck injury.

A bias that would lead to overestimation of the protective effect of helmets appears in a case crossover part of the study, where recall on the previous outing may have been less accurate. However, authors reassure us by saying that they found a practically same helmet effect in the matched case-control analysis, as they did in the case crossover. It is always good when different outcomes and analyses give results supporting the same conclusion.

Confounding

If we find the association between the exposure and the outcome of interest (for example if we found significant differences in injuries between those who wore helmets and those who did not) this doesn’t necessarily mean that one is directly associated with the other. On the contrary, many times the association is due to a third factor, which is associated with both the exposure and the outcome of interest, but independent. We call such a factor—a confounder. When assessing studies, you should always ask yourself whether the authors took confounders into account, and have they analysed their data appropriately.

In this study, in addition to helmet use, the authors asked the participants about what they thought were other potential determinants of head and neck injury: general characteristics (age, sex, ability, experience, lessons, education, and previous head or neck injury) and circumstances at the time of the injury (hours of participation in the activity, damage to non-helmet equipment, self reported speed in relation to average, participation type, mechanism of injury, protective equipment other than a helmet, run difficulty, visibility, snow conditions, and temperature. This seems quite extensive and satisfactory. In the methods section, under statistical analysis, the authors also describe in reasonable detail the approach they used to assess for confounders.

Chance

Even after bias and confounding are taken into account, certain possibility remains of the findings being attributable to (ie, resulting from) chance. In this paper, the extent of this danger is expressed by 95% confidence intervals. Another common way of doing it is reporting a P value. The main point of any study is to extrapolate the findings to populations (ie, conclude about the actual parameter in the population based on the result observed in the studied sample). A 95% confidence interval is a range in which we are 95% sure that the true population value lies. This means that 19 out of 20 confidence intervals will indeed include the true population parameter.



studentBMJ 2005;13:89-132 March ISSN 0966-6494