Leanne Tite takes you through a proof of principle study about whether dogs can detect bladder cancer through the smell of human urine

Abstract

Objective—To determine whether dogs can be trained to identify people with bladder cancer on the basis of urine odour more successfully than would be expected by chance alone.
Design—Experimental, “proof of principle” study in which six dogs were trained to discriminate between urine from patients with bladder cancer and urine from diseased and healthy controls and then evaluated in tests requiring the selection of one bladder cancer urine sample from six controls.
Participants—36 male and female patients (age range 48-90 years) presenting with new or recurrent transitional cell carcinoma of the bladder (27 samples used for training; 9 used for formal testing); 108 male and female controls (diseased and healthy, age range 18-85 years—54 samples used in training; 54 used for testing).
Main outcome measure—Mean proportion of successes per dog achieved during evaluation, compared with an expected value of 1 in 7 (14%).
Results—Taken as a group, the dogs correctly selected urine from patients with bladder cancer on 22 out of 54 occasions. This gave a mean success rate of 41% (95% confidence intervals 23% to 58% under assumptions of normality, 26% to 52% using bootstrap methods), compared with 14% expected by chance alone. Multivariate analysis suggested that the dogs' capacity to recognise a characteristic bladder cancer odour was independent of other chemical aspects of the urine detectable by urinalysis.
Conclusions—Dogs can be trained to distinguish patients with bladder cancer on the basis of urine odour more successfully than would be expected by chance alone. This suggests that tumour related volatile compounds are present in urine, imparting a characteristic odour signature distinct from those associated with secondary effects of the tumour, such as bleeding, inflammation, and infection.

This month's paper is Willis CM, Church SM, Guest CM, Cook WA, McCarthy, N, Bransbury AJ, et al. Olfactory detection of human bladder cancer by dogs: proof of principle study. BMJ 2004;329:712. You can read it by clicking on the following link.

Anecdotal evidence has shown that dogs, with their acute sense of smell, might be able to detect cancers in humans by picking up the odour given off by the cancer cells. In the past 15 years, two letters have been published in the Lancet, seemingly describing dogs' ability to predict the presence of cancer later diagnosed by a doctor. Alongside the anecdotes, good theoretical evidence indicates that dogs at least ought to have such capabilities, because we already know that tumours release volatile organic compounds (or scent chemicals) into the air. Although these compounds are too faint for humans to detect, they could well be within the range of a dog's olfactory (sense of smell) capabilities. In addition to the theoretical benefits, a clinical motive for investigating the possibility of using dogs to diagnose cancer exists, because interest in volatile compounds as indicators of cancer is growing elsewhere in medicine. The purpose of this study was to test, under rigorous experimental conditions, the idea that dogs might be able to detect cancer through their sense of smell.

As compelling as the Lancet anecdotes are, these alone are not enough to convince us that dogs can or even could reliably detect cancers. If you were to sit and think about it for some time you could probably come up with hundreds of equally plausible explanations as to why the dogs in these stories seemed to be able to predict their owners' cancer, including plain old coincidence. But which of the explanations is the correct one? Scientific research, on the other hand, is about the pursuit of truth, or at least in the case of medicine, knowledge that is valid, reliable, and of sufficient certainty to be useful. Scientific experiments allow us to develop this knowledge by testing events repeatedly in a controlled environment, and, in so doing, rule out all the other possible explanations for an event. Under these circumstances we gain far greater certainty about the true cause of an event. A proof of principle study is the first step in scientifically testing what the true cause of an event of interest is, in this case, how dogs seem to be able detect cancer in humans. At this stage, the researchers are not interested in how well, or under what circumstances something works, just simply whether or not it works—clearly a vital first step if time and money might later be invested developing the new finding.

Believe it or not, when scientists design an experiment what they are really trying to do is “falsify” the event that they are interested in. Effectively, they start from the very unbiased position that what they are testing in fact probably does not work. If the test still passes, even after you have made it easy for it to “fail,” then there must be something in it. This approach is no less important for a proof of principle study. Here, the researchers deliberately set up a stringent test, to make it easy for the experiment to fail if in fact the dogs cannot smell the cancer cells in humans. For example, they deliberately avoided selecting dogs that might be particularly good, because of training or breeding, at discriminating different smells. They also made sure that a whole range of other diseases was represented among the control group (or the non-cancerous urine samples). If the dogs are still successful in picking out, or discriminating, the volatile compounds related to cancer, then the researchers can be more sure that it was really was the cancer the dogs were detecting and not some other odour present in the urine.

ROD GILL/REX
Dog detective: can they sniff out cancer?

Designing an experiment that is easy to falsify or to fail if the event you are testing does not actually work in real life is not the same thing as designing a lenient test, however. In fact, designing a good experiment, that allows the test to pass if it works but fail if it does not, needs much careful thought and planning. Notice in the discussion that the researchers considered pooling (or putting into two big samples) all the urine samples from cancer patients and all the samples from non-cancer patients. Even though this would have made the odour signatures stronger (and therefore easier for the dogs to detect) they decided against pooling the samples partly because of the risk that the non-cancerous sample could be contaminated by a rogue undiagnosed cancer case in the control group. So making sure that any factor that could affect the outcome of your study, other than the one you are trying to falsify, is an important part of designing a useful valid experiment.

Notice also that the sample sizes used in the experiment are relatively small: altogether there were six dogs, each of which underwent nine test trials, resulting in just 54 tests or cases for the researchers to include in their analysis. Compare this with clinical trials in which, for example, a drug might be tested on thousands of people before being licensed. But remember that this is a proof of principle study. At such an early stage of researching a new idea, the experiment could just as easily give a negative as a positive result (showing that the dogs in fact could not detect the cancerous samples any better than by chance). In this case then, for reasons of both resource and experimental conservatism, it makes sense to run the minimum amount of tests necessary to test the idea.

Handlers first trained the dogs to recognise the specific odour signature of urine from patients with bladder cancer. Bladder cancer was specifically chosen for the study because of the ease of collecting and storing urine, which would contain the necessary cancer volatile organic compounds needed for training and later testing the dogs.

The dogs were then tested under experimental conditions with new urine samples that had not been used during training. The use of new urine samples in the test phase was vital, otherwise the dogs could have relied on memory to select the target cancer samples. Each dogunderwent nine test trials in which they were required each time to indicate which urine sample out of a series of seven contained the signature odour of the cancer volatile compounds.

In each trial, six of the samples were from patients without cancer. Importantly, the position in which the cancerous urine sample was placed was varied randomly for each trial—had there been any kind of pattern determining where the correct sample out of seven was positioned, the dogs might have quickly learned from and relied on this (and not their sense of smell) to select the right sample. In addition, the dog handlers were blinded (or unaware) to which sample in each trial was the target sample. Otherwise the handlers might have inadvertently signalled to their dogs which sample was the correct one (the dogs would already be highly sensitised to reward signals from their handlers).

Notice that the researchers tried to ensure that for each trial the target cancer urine sample was similar in other important ways to the control urine samples present—for example, by matching the sex of the patients who had given samples in each trial, as well as health, age and other urine characteristics. This gives the researchers more confidence that when the dogs were selecting the correct sample, they doing so because of the cancer odour and not the other odours present in the sample.

Had the dogs been selecting samples randomly you would expect an average success rate of about 14% (or one in seven—because the dogs had to pick one sample out of seven on each trial). The analyses showed, however, that across all dogs, the average success rate for selecting the target sample was 41%. This shows that training the dogs to detect and select cancer odours does improve, beyond chance level, their ability to discriminate urine from patients with bladder cancer from patients without bladder cancer. Even when other important characteristics of the urine were included in multivariate (or multiple variable) statistical analyses, the success rate of the dogs was not greatly changed, indicating that other odours, some of which might be naturally associated with disease, were not determining the dogs' selection of urine samples. The researchers also found that dogs given wet, as opposed to air-dried, urine samples were significantly better at selecting the correct sample.

The strength of this study lies in its simplicity—by keeping the design simple, conservative, and stringent the researchers have presented uncomplicated evidence that is hard to fault, showing that dogs can indeed discriminate odours associated with urine from patients with bladder cancer. Although in its simplicity, the research has left many questions unanswered. But by turning this problem around you can see where the value of simple proof of principle experiments lies—in raising questions and opening up future possibilities. Could this procedure be of clinical significance? What other cancers, or disease might be detected in this way? How reliable is the procedure in larger samples, and what are the limits of its use?

The objective of any good proof of principle study is to raise these kinds of questions that will hopefully one day be answered through future research. Not only does the proof of principle study raise many useful and important questions, but it also serves as a benchmark or starting point that other researchers can draw on and which may one day lead to a new medical procedure capable of changing peoples' lives.

Guidelines: We intend publishing within 24 hours all responses that contribute substantially to the topic under discussion. To avoid points that have already been made please read other people's responses before posting your own. All responses will be eligible for publication in the paper studentBMJ, providing that current appointment and place of work are given. Name and email address are required to send a response and will be published with the response.

Please note: When writing a respnse to a response, please enter the original title of the article and the author of the response you are responding to.