A common fallacy we all make is the appeal to accomplishment. This occurs when we claim that an opinion or idea is invalid because the person making the claim doesn’t have the accomplishments, experiences, or perhaps moral standing to make such a claim. “How can you stand there and criticize the chef’s cooking? Have you ever been a chef?” I prefer, however, to call this fallacy an appeal to experience.
The appeal to experience comes in many forms. For example,
- Your ideas about education are invalid, because you’ve never been a teacher.
- Your ideas about some area of science are wrong, because you’re not that type of scientist/you’re not a scientist.
- Your ideas about a certain disease are wrong, because you don’t specialize in that disease.
- Your ideas about a certain disease are wrong, because you don’t personally have that disease.
- Your ideas about some aspect of sociology are wrong, because you don’t belong to that social group.
I’m sure you can think of more. All of these arguments are logically absurd. You may be wrong about your ideas, and your lack of experience may even contribute to why you are wrong, but you are not wrong simply because you lack the experience. Arguments should stand on their own merits, not the merits of the ones making the arguments. An argument should only be discounted because it lacks merit.
But this fallacy is important for a different reason, and it relates in particular to the practice of anecdotal medicine. Here’s a howardism:
Our experiences lead us to awareness, not insight.
People often confuse awareness and insight. Let’s say you were a victim of a violent crime. This experience might make you a passionate advocate for reducing such crimes and helping victims of such crimes. But your experience alone will not tell you how best to do those things, and, worse, your experience may bias you in a way that you might never be the best person to solve the problem. You might want to focus all of our resources on ineffective measures related only to this one problem that has so viscerally affected you, and by doing so you might actually cause increased crime elsewhere.
Extend this metaphor to healthcare. Someone may be passionate about a certain disease (say, ovarian cancer) because she has been personally affected by it or has taken care of many patients who have died from it. This visceral bias may lead to selective interpretation of data and a desire to do whatever, no matter the unintended consequences, to prevent ovarian cancer. Such passion may lead for the advocacy of screening CA-125 tests or screening ultrasounds, both of which have been found to be harmful. But often it takes an objective observer to see the real risk and benefit ratio of a given intervention.
Our experiences makes us aware of problems, they allow us to have empathy for others, they focus our attention, but they also make us biased in a deep, visceral way, and our experiences may intellectually cripple us, making us unaware of larger, more global issues. The person with experience is often not the right person to make decisions about the things he has personally experienced. When our personal identities are so closely tied to a particular thing, say a way of practicing medicine or teaching or anything else, we may have a problem seeing the forest as we focus too much on our one blade of grass. This is called inattentional blindness.
So how do we gain insight? Through deliberate analysis and study. This needs to be dispassionate and objective, considering all factors fairly. A person with “experience” can do this and her experience can be valuable, but she must carefully check against her own biases. We don’t need to experience a thing to gain insight into it. I don’t need to damage a ureter during a hysterectomy to gain expertise at preventing ureteric injury. I don’t need to perform a series of Zavanelli maneuvers in order to know how to do one when called upon to do so (okay, I’ve done one…).
In medicine, many discussions (a form of argument) with colleagues or patients are begun or ended with “I once had a patient who…” or “In my experience…” or “When I do this procedure…” etc. This may serve the purpose of providing context for the argument and that is fine, but too often these anecdotes are the argument. It is a claim that the experience with that patient or procedure is sufficient to end the debate, and this is the hallmark of anecdotal medicine.
Patients often find this appealing, and many will seek out the most experienced physician, assuming that this is the same as the most competent physician. This makes the assumption that experience is a predicate of insight. Patients often ask, “Have you ever seen a patient with this diagnosis before?” when they mean to ask, “Do you have competency in treating this diagnosis?” These are often not the same question.
One of my favorites (as a male gynecologist) is, “I know you’re a man so you don’t understand what it’s like to have a period….” That’s true, but I am an expert in treating problems associated with the menstrual cycle and I have discussed the menstrual cycles with thousands of women; I’ve read about the menstrual cycle in dozens of books and papers, and I possess the shared and accumulated knowledge of hundreds of years of scientific research. Even better, I’m not personally biased about the menstrual cycle and I don’t believe it causes my toes to hurt, as someone might anecdotally believe who notices that her toes hurt once a month. So while I lack the joy of personally having a period, I am qualified to treat one.
This appeal to experience argument is extended to almost every issue in modern society. Unless you have a child with autism, you can’t decide whether vaccines are related to autism. Unless you are a victim of gun violence, you aren’t entitled to have an opinion about gun laws. Unless you’ve walked a mile in my shoes, don’t tell me how to walk.
But worse than this, the argument in medicine is, “I don’t care what the evidence says, I know from my own personal experience that it works.” I have heard this more times than I care to mention. “I don’t need to read about it, I treat it every day!” “I know the literature says that this doesn’t work, but in my hands I have excellent outcomes.”
For all its faults, evidence based or science based medicine is lightyears ahead of anecdotal medicine. It is odd, to me at least, that most physicians still practice anecdotal medicine. Even those who claim to believe in evidence based medicine tend to pick and choose the literature that confirms their biases and ignore the literature that disagrees with them. This is disingenuous at best and dishonest at worst. But humans, especially doctors, like to believe that their experiences are more valuable than those of other people. It makes us feel special to think that we can intuit how to do something and use our genius, rather than just blindly follow best practices.
When you practice anecdotal medicine, you are basing your patient’s health on a few personal experiences. Whether your past experience with the disease is one patient or 1,000, you are still operating on assumptions based on far fewer patients than our collective experiences of thousands or tens of thousands of patients. What’s more, your patients were “collected” in a biased way, without controls, subject to faulty memories, and the prior patient or patients you recall may not match the current patient in front of you. Even worse, you are likely committing the post hoc logical fallacy, which states that since some second thing happened after some first thing, then the first thing must have caused the second. “I gave her an antibiotic for a UTI and her pain went away.” This is an illogical statement. Her pain went away, yes, but you don’t know that it wouldn’t have gone away anyway if you hadn’t diagnosed her with a UTI and treated her. In fact, for all you know, she might have gotten better even sooner if you hadn’t misdiagnosed her and treated her with an antibiotic. An incredible amount of medical practice is based on this fallacy alone.
People really believe in anecdotes, though. Michael Shermer has said,
Anecdotal thinking comes naturally; science requires training.
I have read numerous defenses of anecdotal medicine over the years. Each of these must, ultimately, take the position that our experience with one or two patients is better than our experience with one or two thousand patients. This quote by Italian surgeon Nicola Basso I think sums up the argument:
The fact that a stone, thrown out of the window, rises instead of falls is not statistically significant, but it is a powerful observation.
I’ve always liked that quote and Basso was a wonderful surgeon. But its total bunk. There is so much evidence that gravity is a thing that if you throw a stone out the window and it goes up, you better start coming up with alternative explanations:
- You are high on psychotropic drugs
- Someone has placed a huge fan under your window
- Its a magnetic stone and there is a huge magnet above your window
- Someone put a string on the stone and is playing a cruel trick on you
Observation of the extraordinary or outlandish doesn’t overturn good evidence. The observation of the stone is not statistically significant for a reason, and this is not a powerful observation, it is a dangerous one. This is the same type of thinking that leads to the conclusion that vaccines cause autism. If the stone really went up, then set up some controls and replicate the finding sufficiently and it will become significant data and we will revise our theories appropriately. But don’t tell me that you ate radish juice for a month and your cancer went away. Until you disprove gravity, I won’t jump out of any windows just because you had a strange experience with a stone.
If you want to explore this type of illogical thinking more, here is a blog that discusses the power of anecdotal proof that vaccines cause autism. If you have the stomach, give it a quick look. The illogic, rhetoric, and cliches are simply amazing. What bothers me, though, is not when this illogic is applied to subjects like autism, but when the exact same logic is applied to things like magnesium tocolysis. Anecdotal evidence is anecdotal evidence, and you shouldn’t criticize this anti-vaxer if you aren’t also prepared to criticize the same logic used everyday in medicine. I see pediatricians everyday take rather self-pious positions about anti-vaxers, and then go to work and practice pediatrics based upon their own anecdotal experiences.
It is true that collected anecdotes may eventually amount to useful data, but we shouldn’t act on that data until it becomes sufficiently collected. Marc Bekoff has said, “The plural of anecdote is not data.” We must collect the data systematically and analyze it blindly and without bias, until a reasonable probability of factuality is reached. A lot of good science starts with a keen observation of something interesting, but a lot of bad patient care also starts with such observations that go unvetted. We don’t apply anecdotes to patients until the information has been replicated, controlled, tested, and validated.
“It is not what the man of science believes that distinguishes him, but how and why he believes it. His beliefs are tentative, not dogmatic; they are based on evidence, not on authority or intuition.” – Bertrand Russell
Of course, we don’t have evidence based answers to many, many questions. This leads us often to decide what to do based on weak evidence or even “expert opinion.” I think many are so frustrated by this fact that they throw away or distrust what evidence we do have and stick to what seems right to them based on their experiences. On the other hand, many EBM purists refuse to do anything for which strong evidence is lacking. Nassim Taleb describes it in this way:
“Doctors most commonly get mixed up between absence of evidence and evidence of absence”
Just because we don’t have evidence that something works doesn’t mean that it doesn’t work. So many things have not been appropriately studied (or cannot be studied ethically given the nature of the problem). In this case, we resort back to experience, but not just experience. We also have analogy, inference, and other powerful tools that can lead us to the best approach. But please don’t do something just because it seemed to work one time before.
For more about anecdotal medicine, read this.