As one who has spent thousands of hours in flight with a parachute strapped to my back or buttocks, I really got a kick out of this article. I’m glad I didn’t realize at the time that anecdotal evidence was just not good enough — most of the anecdotes came from guys who had personally survived an ejection from a disintegrating airplane and their stories were quite convincing. On the other hand, I have friends who are avid skydivers, and they are remarkably tolerant of those of us who would “never jump out of a perfectly good airplane.” To each his own. Aloha,

Ned

recce101

I enjoyed the story, and Ned’s response. However, there is a more serious question. You can be evidence-based (I certainly am) but still think that the standard is wrong. Right now, the assumption is that a treatment doesn’t work unless it meets a variety of tests to prove otherwise. It’s got to show promise in phase I and phase II trials and then show with 95% certainty that it beats a placebo. I would argue that this is not the correct standard in circumstances where there are no other effective treatments.

An analogy would be standards of proof in court. In the US, in a criminal case the standard of proof is “beyond a reasonable doubt”. A different standard is used in civil cases, “preponderance of evidence” (is it more likely than not?). Treatments for metatstatic lung cancer should face a different (and lower) burden of proof than treatments for athlete’s foot.–Neil

neilb

I appreciate both of your comments. First, Ned, I didn’t didn’t make the point that surviving skydivers provide a selection bias by being a retrospective analysis, as you point out. And Neil, I do agree that it’s a mistake to pray at the altar of a p-value of 0.05, saying that a treatment is our new standard if it surpasses that threshold, but not valuable if it fall short of that. Of course, it’s very easy to misinterpret by saying that a non-significant improvement is simply a negative trial. We’ve struggled with that for things like the post-operative carbo/taxol trial CALGB 9633, which may be considered as negative or “not positive enough”.

I’m actually at a lung cancer meeting right now where the experts are debating whether the maintenance chemo trials are really adequate to be alter the standard of care, and this same issue is emerging. Last year at ASCO, the trial by Fidias and colleagues looked at immediate vs. delayed second line taxotere after 4 cycles of first-line chemo, and the trial showed a huge improvement in progression-free survival and a nearly significant improvement in overall survival (p = 0.09), albeit with some flaws: namely, that the first mandated CT on the arm getting chemo delayed until progression wasn’t until 3 months later, which is too long, I think, and there were too many patients who dropped off by becoming too sick for more chemo. We said we needed more data, and then this year another trial with immediate second line (or maintenance) alimta vs. placebo shows a startlingly similar and highly significant improvement in progression-free survival, and a preliminary overall survival benefit of 3 months, for which the p-value was 0.06. This study didn’t include a requirement for treatment at progression, and only 50% of the patients on the placebo arm didn’t get it.

Several experts discount these trials now because they didn’t significantly improve survival, as if the p-values of 0.09 and 0.06 didn’t even suggest a benefit. I think this is a real disservice, and that it’s very appropriate to acknowledge that the evidence is very highly suggestive of a meaningful benefit, even if it isn’t ironclad. Every doctor is forced to make decisions that don’t have a clear answer delivered by great evidence, so you need to use judgment. Turning a blind eye on compelling evidence just because the p-value was 0.06 instead of 0.05 is disingenuous, I think, especially whn I know that these same doctors pursue many approaches that are major departues from the evidence.

Too often, we wrap ourselves in the flag of evidence-based medicine when it supports what we want to do anyway, but we talk about “the art of medicine” when we want to deviate from the data.

-Dr. West

Dr. West

Many doctors (if not ALL doctors) already treat patients based on anecdotal evidence as it is. Whether it be giving additional chemotherapy past 6 cycles for cancer treatment, or just giving cisplatin for SCLC, which has not been approved by the FDA for that specific cancer (if it was effective, why hasn’t the FDA approved it’s use for this lung cancer), treatment bias based on these physician’s personal experiences occurs every single day.

This built in anecdotal bias was brought to my attention recently when my 1 year old caught had a terrible run a stomach rotovirus (which kills 611,000 children around the world every year.) I came to later learn that this rotovirus had a vaccine available for it for other sub-types of the virus, so I was interested in getting my daughter vaccinated to protect her from the other subtypes of the virus. Before we could get the whole sentense out of our mouth asking the doctor about the vaccine, the doctor said “Absolutely Not!”, and that one of her colleague’s children almost died from the vaccine.

I later went home to read up on the vaccine, and found that it was never proven that the old vaccine, taken off the market years ago, that it was never proven to be the case of 1 in 12,000 bowel obstructions that occurred. However, there was a newer and safer vaccine on the market since 2006, seemingly without any evidence of bowel obstruction, and our pediatrician refused to offer it to us.

This year, another newer rotavirus vaccine has also been approved, however the bias due to the old vaccine, is STILL a factor on whether or not a pediatrician would use it.

http://content.nejm.org/cgi/medline/pmid;17200265?hits=20&fyear=1997&where=fulltext&tmonth=Dec&fmonth=Jan&tyear=2006&searchid=1&FIRSTINDEX=0&resourcetype=HWCIT

“A network of 431 pediatricians ……….Respondents were asked about intentions to use the vaccine and anticipated barriers. RESULTS: The survey response rate was 71%. Of the respondents, 52% strongly agreed and 37% somewhat agreed with the need for a rotavirus vaccine. If recommended for routine use, 50% would strongly recommend and 34% would recommend but not strongly; 52% would begin to use within 6 months and 27% from 6 months to 1 year. ”

So only a 50% immediate adoption rate of a rotavirus vaccine, which is undoubtedly a result of the past unproven and rare anecdotal evidence of bowel obstruction incidence in an older, now off the market, and completly unrelated vaccine. They are not practicing based on the science, they are practicing based on their guts, otherwise there would be a 100% adoption immediately.

This is of couse unrealted to cancer, but just proving my point that contrary to popular belief, that many doctors actually conduct their practices based on anecdotal evidence every single day. Science and statistics don’t always drive actual clincal practice.

Jim

dadawg001

Jim, and everyone else,

Yes, I absolutely agree that practice patterns are a combination of the evidence and our interpretation, which is full of our own prior experiences and different levels of risk-aversion. In discussing people’s biases and other motivations, I realize that there are some situations where you simply can’t provide enough evidence for people to change their mindset — it’s like arguing religion. Obviously, if it were just about the evidence, we wouldn’t have these differences based on our preconceived notions, but that’s the way it is. It’s also why drug companies spend more money on marketing medicines than on research and development. In a perfect world, the evidence would allow a good drug to sell itself, but that’s not what we’re seeing…And many times, the drugs that are marketed the most heavily are the ones that need the most help to convince people of its value (I’d say “to convince doctors”, but a huge amount of this field is now direct to consumer advertising, which tends to focus very significantly on the drugs that are of marginal value, that NEED to get people to ask for them, like putting candy bars in the check-out aisle at the grocery store; they never have to entice you with the stuff you really need).

-Dr. West

Dr. West

Dr. West,

This is the first time that I’ve heard a good argument FOR the advertising of medicines that we are all inundated with on a daily basis (TV commercials, free trials at the Dr’s office, magazine ads, etc.)

I didn’t think of the possibility that the purpose of these ads might be to drive the consumer (or patient) to force or educate their doctors to change some of the bias that they have accumulated through years of practice (even if the bias developed unwittingly.) I guess doctors are human too.

And I guess these drug advertisements, to a certain extent, do have a valuable purpose in medicine and making patients knowledgable about new treatment options available to them.

I wonder though, how many lives might be able to be saved each year if we could strictly stick to the science, and eliminate the bias?

Jim

dadawg001

Jim,

Actually, the idea of providing a really good education to the patients and their families, so that they could know the best options and discuss them with their doctors, is exactly why I wanted to start OncTalk/GRACE. But I wanted it to be a more thorough discussion of the evidence for and against a treatment, not just a 30-second spot that too often misleadingly suggests unrealistic benefits from a treatment. We rarely see people dancing through fields from our cancer treatments, even though the TV ads would have us think that many patients spent countless hours doing this.

But the big picture is that I absolutely believe that we can improve medical care by educating patients and their families. You just can’t do a topic justice in a brief commercial.

-Dr. West

Dr. West