Paper plus: Public smoking bans and heart attacks

Leanne Tite takes us through an opportunistic before and after study which investigated whether a smoking ban was associated with lower incidence of admissions due to acute myocardial infarction

This month's paper is Sargent RP, Shepard RM, Glantz SA. Reduced incidence of admissions for myocardial infarction associated with public smoking ban: before and after study. BMJ 2004;328:977-83.

What is the study about?

Laboratory and other scientific studies tell us that exposure to secondhand cigarette smoke causes biochemical and physiological changes within the body associated with acute myocardial infarction. Other studies have shown that the risk of acute myocardial infarction increases by up to 30% after regular exposure to secondhand smoke. It follows then that by reducing exposure to secondhand smoke, the risk and general incidence of acute myocardial infarction should also decrease. One way to achieve this might be through a ban on smoking in public places. The current study attempts to test the link between public smoking bans and the incidence of acute myocardial infarction by comparing the number of hospital admissions for acute myocardial infarction before and after a public smoking ban was implemented in an isolated community in Montana, USA.

What evidence did the researchers look at?

This kind of study is an opportunistic study, in which the researchers collected evidence about a naturally occurring event. The advantage of these kind of studies, which are sometimes referred to as "natural experiments," is that researchers can look at relationships between events that cannot be set up in a laboratory, perhaps for practical reasons or to maintain ecological validity. This is often the case with policy research or studies which look at events within a specific community or geographical location. In such cases, a natural experiment is often the only means available to the researcher to study the problem under question.

In this study, the researchers used hospital records to look at the number of diagnoses of primary and secondary acute myocardial infarction during a six month smoking ban in a small community in Montana, USA. To see if the ban was associated with a change in the number of diagnoses, they compared data from the ban period with data from the same six month period in each of the five years preceding the ban and one year after the ban was suspended. Thinking in experimental terminology, we can say that the data collected during the ban period can be thought of as the study group, and the data collected before and after the ban can be thought of as the control group. The "before," or control group data is important because it serves as a benchmark against which acute myocardial infarction figures from the ban period can be compared to look for any change.

One advantage of this study is that the community where the smoking ban was applied is geographically isolated and served by just one hospital. This implies that residents might be less likely to spend time outside of the ban area, and in case of illness would be most likely to attend the local hospital from where the acute myocardial infarction records were taken for the study. This is important because it means that the impact of the public smoking ban on residents' health should be better confined within the community and also more likely to be picked up in the local hospital records. If the ban covered a more dispersed community, its impact might be obscured if residents spent a significant amount of time in places where public smoking continued.

What are the limitations of natural experiments?

The use of the term "natural experiment" to describe a study like this is perhaps a little misleading because many of the rules of true experimentation are violated when naturally occurring events are studied. Essentially, experimental methods follow what is known in philosophical terms as the method of difference. This means that a test is applied twice, either once to two groups or twice to the same group, which is identical on both occasions except in just one respect. Any difference in test outcome can then be attributed to the difference in treatment conditions and a causal link can be confidently drawn to explain the difference in the outcome of the two tests. Of course, this kind of approach requires controlling all factors across the two tests except for the change you are interested in. In natural experiments this is always impossible because there are many factors, both known and unknown, operating at any one point that could be influencing the outcome of the variable you are measuring. This does not mean that natural experiments are a waste of time, but that they are really only useful for looking at associations between variables.

When two variables are associated it is tempting to assume that they are causally related, especially when it seems very logical or intuitive. However, it is always a mistake to make a causal link based on an association alone. For example, suppose a researcher finds an association between secondary school exam grades and truancy so that exam grades are better for those who truant less. You might assume that playing truant impairs exam grades, but it could also be the reverse, that poor exam grades lead to increased truancy. On the other hand, it could be that both poor exam grades and truancy are conjointly caused by a third factor such as poor teaching, or the association could be completely spurious.

In the current study you will notice that the researchers compared the six month ban period with same six month period in the five years preceding the ban as well as one year after. The reason that they looked at only the same six month period and not the whole year in the "control" years was to attempt to make the comparison more valid. Remember that in any study the control group should be identical, or as close to the study group as possible. By looking at the same six month period, the researchers will have controlled, to a certain extent, any systematic bias that might be associated with either smoking or acute myocardial infarction incidence at certain times of the year. By studying several years before and after the ban period they would also hope to reduce the impact of random changes in smoking or heart attack rates in any one year period.

What did the data show?

The figures collected from the local community hospital show that there was a 40% decrease in acute myocardial infarctions during the six month ban period compared with the control years when no public smoking ban was in place (from an average of 40 a year before and after to 26 during the ban period). Although the authors rightly describe their paper as a study of the association between a public smoking ban and the incidence of acute myocardial infarction, there still seems to be an implicit message throughout the paper that the reduction in incidence during the ban period is attributable to a reduction in secondhand smoke in public places. This conclusion is not justified because the researchers did not directly measure changes in secondhand smoke prevalence in public places, nor look at changes in smokers' behaviour during the ban period.

What are the problems with the study?

One of the main problems with the study is that the number of people having acute myocardial infarctions in any of the years studied was quite small (an average of 40 during the six month period). This means that even a small change in the incidence of acute myocardial infarction in any one month or year would look like a statistically significant and so meaningful change. But it is not inconceivable that acute myocardial infarction incidences fluctuate on a monthly and yearly basis for any range of reasons. This makes it even more difficult to argue that the public smoking ban was responsible for the observed reduction in acute myocardial infarctions during the ban period. As an illustrative point, notice that the incidence of acute myocardial infarction outside of the ban area increased during the ban period by six incidences from 12 to 18 (see the table in the paper). Although this change did not register as statistically significant, it nevertheless represents an increase of around 50% which does sound quite meaningful. This goes to show that whilst the results of statistical procedures are important, they are meaningless without reference to the numbers and the quality of the data that are being tested.

COURTESY OF INDEPENDENT RECORD

Newspaper announces Montana's smoking ban
Conclusion

Given the issues with natural experiments described above what conclusions can be drawn from this research? The data collected does seem to suggest that the smoking ban had quite an impact on the incidence of acute myocardial infarction. But without a careful examination of smokers' behaviour and smoke concentrations in public places during the ban period, it is impossible to attribute this change to any one causal factor. It could be that the ban was instrumental in decreasing morbidity due to acute myocardial infarction, but perhaps smokers simply spent more time outside of the ban area during the ban period so displacing the problem, and suggesting that a widespread public smoking ban might not be effective. Given also the small number of incidences of acute myocardial infarction during the study, being sure of how much of the observed change was due to random or other unknown factors is difficult. In their conclusion, the researchers state that "laws to enforce smoke-free workplaces and public places may be associated with an effect on morbidity from heart disease." This is absolutely true, quite conceivably such bans "may" have this effect. But given the various pitfalls of the research, the question is whether we are any more convinced of this than we were before the research was published? Unfortunately, I think not.

Leanne Tite, researcher, BMJ
Email: ltite@bmj.com


studentBMJ 2004;12:221-264 June ISSN 0966-6494