From the GRACE Archives - Originally Published November 21, 2009 | By Dr. Weiss
I’ve been impressed by many things about GRACE and its members. Among these many things are the thirst for new information and better treatments. I also admire the openness of the members both to new ideas, and to questioning these ideas.
Some new ideas succeed and, after careful evaluation, truly advance care. Others fail. As a clinical trialist, I regularly evaluate ideas to consider a clinical trial. Discussion with cards7up led me to realize that it might be helpful to share this thought process with this group—on behalf of yourselves and your loved ones, you vet ideas from the popular media, friends, family and the world wide web. I hope that these thoughts can help you to do so with more confidence. (The original thread begun by cards7up is pasted below, after the comments)
This slide is from a talk that I gave a few weeks ago to a patient group in Philadelphia. It shows the four basic sources of ideas for a trial:
• Laboratory data
• Clinical observations
• Alternative Medicine
• Successes in other cancer
Some people might be surprised to see alternative medicine as an idea source. It is a common misperception that Western doctors are not open minded to alternative medicine. Taxol started as the bark of the Pacific Yew tree, ginger is gaining acceptance as a nausea treatment, and acupuncture has found a variety of roles in mainstream medicine. What I am closed-minded to is treating patients routinely with something whose efficacy and safety are unproven. So, new ideas are good, as long as they’re tested properly. We first take a new idea to the lab, and learn everything we can about how it might work. We test it on cancer cells in a dish. If it’s the next cure for cancer in a dish, we take it to animal testing where we test for efficacy, safety and dose. If it looks like the next magic bullet for rodent cancer (again, I can’t tell you how many times we’ve cured cancer in various rodent models) then the idea can start the clinical trials vetting process.
The vetting of a trial starts with a funding source. The investigator writes a proposal to the NIH, a pharma company, or a foundation laying out all of the data leading to the proposed trial and explains the basic structure of the proposed trial. If the funding source finds the idea promising and chooses to fund it, the investigator will next write a protocol. The protocol is a detailed guide for how a trial will be run. Once completed, it is submitted to a scientific review board. The job of this committee is to evaluate if the idea has sufficient scientific merit and promise and to ensure that the clinical rationale is sound. If they sign off, it next goes to the institutional review board (IRB) who ensures that the study is ethical and protects human subjects. If they approve the study, it may open, with either a data safety monitoring board (for large trials) or a medical monitor (for small trials) monitoring the trail to ensure subjects’ protection. Finally, at completion of a study, the results, positive or negative, should be presented and published.
Okay, so that’s how an investigator thinks. What about a practicing oncologist faced with an onslaught of new ideas published in various journals, presented at various conferences, and promoted in the media? How does a doctor, working a gazillion hours a week seeing patients choose what to read and what to believe? How does he or she evaluate a paper?
First, it is key to understand the idea of levels of evidence. Different ideas about how to treat patients have different levels of proof behind them. I’ve summarized these in the trial ladder below.
My next major point also answers the question of why your oncologist has never heard of the great new cancer therapy that you read about on the internet, or even the news. Given finite time, we want to read (and you should value most highly) the best, most proven papers. How do we know which ones they are? There’s a hierarchy of prestige levels in the cancer journal world, and authors try to get their papers accepted to the best possible journals. While I’m sure that oncologists will argue about which ones are best, , here’s my opinionated, very rough guide for clinical trials in lung cancer:
• Top tier: Journal of Clinical Oncology (JCO), New England Journal of Medicine (NEJM), Lancet, International Journal of Radiation Oncology Biophysics (“Red journal”)
• Second tier: Journal of Thoracic Oncology (JTO), Chest, Lung Cancer, Journal of the National Cancer Institute (JNCI), Clinical Lung Cancer, Journal of the American Medical Association (JAMA)
Often worth of reading: Other peer-reviewed, medline-listed medical journals
There’s also a hierarchy of conference presentations, with the American Society of Clinical Oncology (ASCO), the European Society of Medical Oncology (ESMO) and the World Lung Conference at the top. Note that findings usually first come out at conferences, then are published in journals — it should raise a red flag if a study is presented but never published.
So, to summarize, it might be worth reviewing brief criteria to know if an idea is worthy of routine implementation. We read high-quality journals. We then evaluate the findings presented and mentally place it somewhere on the trial ladder shown above. Finally, if we’ve accepted the idea that the data is true, we then start the more comprehensible process of asking if the difference is big enough and if it’s worth the toxicity.
In the end, properly vetting an idea is challenging even for trained oncologists. We meet in journal clubs to discuss new findings and we discuss them with colleagues. Those more pressed for time can turn to review articles and conferences. In these media, trusted experts summarize the data and give their interpretation. Time consuming? Yes. Difficult? Yes. But the stakes are high—failure to move a promising idea forward denies better therapy to patients in need. Treating patients with untested therapy denies them more promising therapy, risks toxicity, and denies the opportunity to move properly-vetted concepts forward to advance care.
Comments closed |15 Responses to How to vet a treatment idea
Comments are closed | 15 Responses
November 21, 2009 at 9:59 am
Thank you for an interesting and informative post. Though we are all anxious for the next breakthrough treatment, your post helps me stay grounded amid the media hype and “not ready for prime time” preliminary results of new therapies.
It also makes me appreciate just how difficult it is for oncologists to stay at the breaking edge of new treatments without adopting a new idea to soon or too late.