My kids are right in the middle of that time when they watch SpongeBob and see commercials for toys, cereals, and music, nearly every one is puncuated at the end with, "Daddy, can we get that? I want that." There comes a time in everyone's life, hopefully early on, when we learn that we won't actually find eternal bliss with every advertised item. Juicy Fruit gum loses its flavor in about 10 seconds, Pepsi doesn't actually make you hip, and the urinating feature of the Betsy Wetsy doll actually loses its appeal (though anyone who changes real diapers probably don't see the appeal of paying for this feature). Learning that advertising makes things seem better than they really are is a truism we all need to learn.

The harsh reality is that many press releases and newsroom puff pieces about supposed cancer breakthroughs are pretty akin to TV commercials. Some represent true, meaningful benefits, but there are a few valuable rules that can help us determine which ones are more sizzle than steak. Many of these stories are planted to manipulate people, specifically to generate buzz, to lead doctors to change their prescribing habits or to get patients to ask for a particular treatment. Here are two key caveats worth bearing in mind to help discriminate fool's gold from the real deal:

1) Beware of results from phase II data being highlighted and compared with evidence from larger trials to suggest that the new treatment is better. These are everywhere. Smaller trials with a few dozen patients from 1-2 centers very typically have unusually fit patients and very typically show better survival results than trials of hundreds of patients from all over the country or world. Phase II trial results are routinely highlighted as being superior to current standards (which are generally based on large trials), with the implication that this is a new better treatment (for example, the press release about a phase II trial of carbo/taxol (paclitaxel) with the "vascular disrupting agent" ASA404 described in a prior post, or a very recent news story about the single-arm SWOG 0536 trial of carbo/taxol with both erbitux (cetuximab) and avastin (bevacizumab) (abstract #9 here). These are certainly promising leads, but the vast majority of these approaches don't end up performing as well in large phase III trials as the did in the preceding phase II work. Some actually still proved to be beneficial, such avastin added to carbo/taxol (abstract here), but even here the carbo/taxol/avastin arm had a survival of 12 months instead of 18 months (at the same avastin dose) in the preceding phase II trial (abstract here). Other concepts have looked extremely impressive as phase II trials and then failed completely when tested in a larger setting (see posts here and here about consolidation taxotere (docetaxel) after concurrent chemo/radiation for an example of disappointment after high expectations).

The issue with such trials is that the smaller, earlier trials involve a good deal of selection bias, which is the idea that the patients enrolled on such trials have features that differ from those of the broader lung cancer population. The smaller the trial, the more likely it is that results will vary from what we could really expect from a large clinical trial. You never hear about to thials that emerge as especially unfavorable (because they're either never published or certainly not publicized), but the ones that just happen to turn out unusually well are highlighted or sometimes heavily over-promoted. At the same time, we know that patients at major cancer centers are typically younger, more fit, and have more resources than the general cancer population. Not surprisingly, results from the most renowned cancer centers tend to produce results that are notoriously difficult to replicate in large multicenter trials.

Consequently, smaller trials from 1-2 centers can provide results encouraging enough to lead to larger follow-up studies, but we often get burned by presuming that these small trials will definitely hold true with further research.

2) It's a mistake to compare results of one trial with those of another, because differences in eligibility criteria can lead to important shifts in expected outcomes. For example, the very commonly used regimen of carbo/taxol produced a very typical median survival of eight months for advanced NSCLC in the pivotal phase III ECOG 1594 trial (abstract here) -- that's very average, quite anticipated for a platinum-based doublet in a large population. But in the later ECOG 4599 trial of carbo/taxol vs. the same chemo with avastin (abstract here) had a survival of over 10 months with the same regimen and in the same setting, but the latter trial excluded patients with squamous cancers, brain metastases, a marginal performance status, and several common medical problems. The more stringent eligibility produced a more cherry-picked trial population in which even the patients who didn't receive the trial drug performed better than history would have predicted.

Similarly, trials of patients with "advanced NSCLC" in the US include patients with stage IV (metastatic) disease and also patients with a malignant pleural effusion, or cancer cells in the fluid around the lungs, which is technically stage IIIB, but a specific subgroup generally termed "wet IIIB" that is standardly treated the same as metastatic NSCLC because we really can't feasibly treat it with curative intent. Outside of the US, however, there are patients with "dry IIIB" disease, who don't have a malignant pleural effusion and who can be treated with curative intent (usually a combination of chemo and radiation), but in Europe and some other parts of the world they are often allowed onto trials of chemo only. Unfortunately, this essentially amounts to undertreatment, because they aren't treated with curative intent, and it's frustrating to others of us in the lung cancer community to see people not offered a proper opportunity to be cured, even if the odds are smaller than we'd like. But the point is that patients with dry IIIB NSCLC have a better survival than patients with stage IV or wet IIIB NSCLC, so European or worldwide trials of advanced NSCLC in which better prognosis dry IIIB patients constitute 15-20% of the population will show better survivals. Median survivals of 10-14 months have been seen in trials like the Eli Lilly-sponsored one of cisplatin/alimta (pemetrexed) vs. cisplatin/gemzar (gemcitabine) (abstract here) and the AVAiL trial of chemo with or without avastin (abstract here): these are trials that accrued largely or entirely in Europe and included some patients with a better prognosis than would have been permitted in a trial that included only those with truly advanced disease. Similarly, many Asian trials show median survivals that can now be in the 18-22 month range, largely because there are many patients with a form of NSCLC that acts like a different disease and can be stunningly responsive to oral EGFR inhibitors like iressa (gefitinib) and tarceva (erlotinib). We now know that you can't presume that trials done in Asia will demonstrate similar results in North America or elsewhere.

The point is that press releases or pharmaceutical representatives may suggest that since a survival of 12 or 13 months in trial X is the same or better than a survival of 10-11 months in trial Y, it follows the treatment in trial X is better. But if trial X included patients who were likely to do better anyway, you can't draw any conclusions without studying these options in the same randomized trial. These comments are marketing, not science, although sometimes it's physicians who lead us down this primrose path.

To make this issue concrete, it's been suggested by some (typically people with a financial stake in alimta) that since cisplatin/alimta in one of the trials above had a median survival in patients with adenocarcinomas that was very similar to that of the carbo/taxol/avastin arm on the ECOG trial (about 12 months for both), that cisplatin/alimta is a way to get the same benefits without the cost or potential side effects of avastin. In what appears to me to be retaliation, Roche announced last week (see here) that the median survival among patients with adenocarcinomas on the ECOG 4599 trial was actually 14 months. Nevermind that this was just an unplanned subset analysis, and this is despite the fact that they don't highlight the results of other subset analyses that didn't look favorable for avastin. I see this as rather aggressive, slick marketing -- basically a race to the bottom. It's particularly unfortunate because Genentech has long prided itself as being a scientific company of very high integrity, but now that it's being bought by Roche, I'm afraid all bets are off. To me, these media announcements and market positionings are like watching political candidates throw dirt at each other. You should be wary of this stuff, because it's a sucker bet.

-----

I'll close by saying that my goal here isn't to crush hope but just to provide the tools of healthy skepticism, some background so that people can judge some of the press releases that are offered to generate irrational exuberance. Some of the smaller trials do pan out to become legitimate steps forward, and some regimens are clearly better than others. But when information is offered, it's important to consider the source and remember that if it seems too good to be true, it probably is.