Well, that was embarrassing, wasn’t it? The prestigious Harvard teaching facility Brigham and Women’s Hospital had to apologize for its study suggesting a potential cancer risk from consuming the artificial sweetener aspartame.
In fact, BWH even admitted that their research on this risk was “weak”. Ooops.
Unfortunately, Brigham & Women’s Media Relations office had already sent out a news release titled “The Truth Isn’t Sweet When It Comes to Artificial Sweeteners.” This is the kind of cheesy news release title, by the way, that is used because it virtually guarantees media pickup by grabbing the attention of reporters and their editors. Then the media repeats the cheesy headline over and over until the rest of us see it so often that we believe it to be true.
As reported by Robert Bazell of NBC News in his post called “Harvard Hospital Admits It Promoted Weak Science on Aspartame “:
“The (study’s) conclusion was so weak that the researchers had to submit it to six journals before they found a seventh, the American Journal of Clinical Nutrition, that would publish it.
“Few reporters read that journal. If it was not for the frightening headline, no one would have known about this study.
“After being asked some hard questions – and just before the report was to be released – the hospital changed its tune, issuing a statement that said:
“Upon review of the findings, the consensus of our scientific leaders is that the data is weak, and that BWH Media Relations was premature in the promotion of this work. We apologize for the time you have invested in this story.”
Well, you just know that the folks who make soft drinks loaded with aspartame were going to be all over this kind of admission.
And sure enough, in between high-fiving each other at this unexpected stroke of luck, the industry umbrella group known as the American Beverage Association (ABA) issued the following statement on the very same day that the BWH apology came out:
“The (BWH study) authors said it best: their study has ‘limited application’ and their findings may be ‘due entirely to chance.’ We agree.
“The fact is: Aspartame, which is an ingredient found in many beverages as well as thousands of foods, has been deemed safe for decades by the world’s leading toxicologists, as well as the National Cancer Institute and other regulatory agencies and public health experts around the world. Let’s stick to the facts.”
No matter what your personal opinion of this artificial sweetener might be (and no doubt whatever you believe has been duly influenced by what you’ve learned in the media anyway), Bazell’s report tends to side with the ABA:
“Aspartame is one of the most studied food additives ever – and with good reason. It is an ingredient in some 6,000 products, but its main use is to sweeten diet sodas. Americans drink an astounding amount of diet soda, the equivalent of 43 billion 12-ounce cans a year.
“Most animal and human studies have given aspartame a clean bill of health. But in 2005 an Italian study showed a potential danger in rats that led epidemiologist Dr. Eva Schernhammer and her team at Brigham and Women’s Hospital to look through the records of more than 77,000 women and 47,000 men in their nurses and health professional’s studies. They concluded that those who drink a daily diet soda sweetened with aspartame could have an increased risk of leukemia, lymphoma or non-Hodgkin’s lymphoma.
“But there are caveats: The results differed between women and men, and there also seemed to be a risk among people who drank mostly sugared soda. No one claimed that it meant more than further study was needed.
Yet when lead author Schernhammer was asked whether the new research proves that aspartame is dangerous, she answered emphatically:
“No, it does not.”
You might ask why BWH Media Relations office staff were the ones to eat crow and apologize for publicizing research results that probably shouldn’t have seen the light of day in the first place. You might also wonder why Dr. Schernhammer et al kept submitting an admittedly “weak” scientific paper over and over to journals who kept turning it down for publication. You would think that the average brainiac might suspect problems with poor quality research methodology.
We won’t even ask why the American Journal of Clinical Nutrition accepted for publication a self-described weak study that six other journals had already refused to publish.
The study’s conclusion read:
“Although our findings preserve the possibility of a detrimental effect of a constituent of diet soda, such as aspartame, on select cancers, the inconsistent sex effects and occurrence of an apparent cancer risk in individuals who consume regular soda do not permit the ruling out of chance as an explanation.”
Read those last few words one more time, essentially saying that pure chance itself might be responsible for select cancers. But each of the study authors listed on this paper gets bonus points on their respective CVs for getting another journal article published.
I guess the thinking might go like this: even if you believe you have a useless study, you might as well keep shopping it around until some editor, somewhere, somehow finally bites. Yes. There’s embarrassment aplenty in this story.
Not all science deserves publicity, adds Bazell. Some research is not done well, he explains. Some comes to equivocal conclusions and serves solely to alert other researchers of the need for further study. The BWH research about a potential cancer from aspartame falls squarely in that second category.
As I’ve reminded readers here before:
“For every PhD, there is an equal and opposite PhD.”
Consider the studies that this week may proclaim: “Coffee causes cancer!” But just wait: next week, yet another study will insist: “Coffee prevents cancer!”
When weak studies do get media attention, they can often increase the confusion and anger that many people feel about science in general – and the study of possible risks and benefits of our diet, in particular.
As cardiologist Dr. Steven Nissen, chair of the Cleveland Clinic’s cardiovascular medicine department, told NBC News (with more charity than some might be able to muster):
“Promoting a study that its own authors agree is not definite, not conclusive and not useful for the public is not in the best interests of public health.”
.
The Ethical Nag 
“But the entire team gets bonus points on their respective CVs for getting another journal article published”
That sums it up.
Another observational study where thousand of p values are calculated, or hundreds, you have to find some significant ones.
A huge proportion of research papers nowadays are as weak as this one. They only serve to feed the media and authors.
If you add the corrupt research, the biased research, dry lab research, aooch and yikes !
Thanks for your perspective here, Pedrinha.
Interesting and balanced view here. Thanks for this. Personally I quit aspartame (e.g. all diet sodas) years ago when I first heard of potential “dangers” in the media. This article is a good example however of how sensational media headlines can actually be based on very flimsy “science”. There are many heads that should roll at BWH over this mess. And at the journal, too.
Agreed. It’s yet another tiresome example of how “sensational” headlines sometimes originate from the least likely sources, e.g. not just The National Enquirer . . .
I’m a bit late to the party on this one, but it’s worth noting that the reviewers didn’t *read* the sentences about chance that you quote; they effectively *wrote* them for the authors of the article, as a condition of publication. See the NPR piece. I’m not sure which is more head-scratching: that the authors might have submitted and resubmitted a paper with findings they knew to be weak, or that they didn’t realize this. Surely it’s the former: They had to pick up some kind of clue from the 6 previous rejections.
Thanks for that NPR link and the important clarification i.e. that the journal ‘pushed back’ to make sure that the ‘chance’ comment was included.
One other thing on this topic (which has semi-obsessed me, both for the science and the reporting). Nissen was much less charitable elsewhere, it turns out. See this piece from WHDH in Boston, where Brigham and Women’s is located:
Two quotes:
“The study itself is not done well, the findings are not reliable.”
And:
“We are in the position of promoting public health and so we put out bad science, we’re actually harming the public’s understanding of what they need to know and that’s not a responsible thing to do.”
“Bad science”! That’s not a phrase you hear researchers use very often when talking about the work of other researchers.
FWIW, I’m not sure I’d go that far. But it’s noteworthy that reporters couldn’t find any outside experts who would say good things about the study. Not the usual, “This is a remarkable study,” or “This is an important finding.”
–Ed
Good point, Ed. Perhaps professional courtesy keeps other academics from slamming “bad science” when it hits them right in the face? I congratulate Dr. Nissen on his honesty. But “bad science” affects the rep of all scientists. They are precisely the ones who should be screaming blue murder at “research” like this instead of merely a wimpy “blahblahblah” (as we translate corporate-speak in the PR field!) BTW, if you want to become even more semi-obsessed, go visit Ivan Oransky’s Retraction Watch site. All they do there is list the latest in published journal articles that have since been retracted due to plagiarism, faked peer reviews, flawed research, scientific misconduct, etc etc. It’s enough to make you cynical . . .
Ooh, that’s fun! I had stumbled on Oransky’s Embargo Watch, but didn’t know about Retraction Watch. I’m a recovering academic, and I watch retractions with a heavy heart. They are still pretty rare; I think rates of fraud and plagiarism have gone up in part because journals and readers have much better tools for detecting it.
I agree with you that scientists should be more forthright in their assessments of others’ work — civilized, but honest. Science is supposed to be a self-correcting process. And it is, most of the time. Self-corrections might happen more quickly if scientists were less nervous about criticizing others’ work. But the incentives often work against this: Everyone in a field eventually reviews everyone else’s work (exaggeration, but roughly true), and no one wants to get into hot water with potential reviewers of their journal articles or grants.
Thanks for the pointer to Oransky’s other blog!
“Dr. Eva Schernhammer and her team at Brigham and Women’s Hospital to look through the records of more than 77,000 women and 47,000 men in their nurses and health professional’s studies. They concluded that those who drink a daily diet soda sweetened with aspartame could have an increased risk of leukemia, lymphoma or non-Hodgkin’s lymphoma.”
Why the heck didn’t they suspect the bubbles ?
Bad bad science, bad bad peer reviewing.
But a paper published in the pocket !
It’s the bubbles, idiot !
Bubbles? Really?
Yeap ! Ironically.
Just to underline the authors’ lack of scientific spirit. If aspartame and sugar sweetened sodas equally are associated ( no possible inference on causation) with some harm, logically they should think about something else.
But no, they have aspartame entrenched as the (media friendly) culprit so they try and prove their point.
A real scientist will build a hypothesis, then try and DISPROVE it. Failing to do so supports said hypothesis.
Maybe the issue is drinking it out of a glass? Or the can? Or using a bendy-straw? Or because it’s Wednesday?!?
Definitely the bendy-straws. Wednesdays may also be a culprit. Interaction between bendy-straws and Wednesdays, p < .05.