Be suspicious of research presented at scientific meetings

I’m not a scientist. I’m merely a dull-witted heart attack survivor who started asking questions about the fistful of cardiac drugs I now have to take each day.  But I did spend 20 years of my life living with a scientist, which meant countless scintillating breakfast table conversations on topics like zinc and copper sediment in the Fraser River estuary. (Does that count at all?)

One thing I did learn from such scintillation is that there’s research – and then there’s research.

Or, as New York Times journalist Andrew C. Revkin, author of Global Warming: Understanding the Forecast, reminds us:

“For every PhD, there is an equal and opposite PhD!”

This may help to explain why we can read in breathlessly urgent news headlines that coffee causes cancer, yet the very next week we’ll read that coffee, paradoxically, prevents cancer.    .

Now it so happens that I have recently returned from covering the Canadian Cardiovascular Congress in beautiful Vancouver, where over 700 scientific papers were presented to delegates.

This is in fact the nature of scientific meetings like medical conferences: hundreds of very smart people present the results of their studies to their very smart peers. And as with most major conferences, each day’s presentations in Vancouver were well reported by local, national and international media attending the proceedings – many prefaced by breathlessly urgent news headlines.

Most media coverage reported the results of these studies as if they were indeed the gospel truth.  In fact, this kind of coverage is generally common whether the quality of research is good or not-so-good.

So here’s a little crash course on why you should be generally suspicious of research presented at all scientific meetings:

  • information presented in talks at scientific meetings may be incomplete
  • research abstracts presented at scientific meetings often receive substantial media attention before the validity of the work has been established in the scientific community
  • many of the abstracts receiving media attention have weak designs, are small, or are based on animal or laboratory studies
  • this work is generally not ready for public consumption: results change, fatal problems emerge, and hypotheses fail to pan out
  • 25% of these studies remain unpublished more than three years after the meeting
  • high-profile presentations that receive front-page coverage are no more likely to be published than abstracts receiving less prominent coverage
  • results are frequently presented as scientifically sound evidence rather than as preliminary findings with still uncertain validity
  • most findings have not undergone final peer review, have yet to be independently vetted, and may change
  • unlike at scientific meetings, evidence strength of research published in peer-reviewed journals can be evaluated
  • physicians, confronted with preliminary research findings, must be able to answer some fundamental questions such as: What is the rush?”
  • develop a healthy skepticism about the ‘breakthroughs’ you repeatedly encounter in the news
Sources: Canadian Women’s Health Network and Health News Review


Find out more about how research works:  Nagging 101

And here’s a small sampling of my coverage of the studies presented at the Canadian Cardiovascular Congress for my Heart Sisters blog readers:


2 thoughts on “Be suspicious of research presented at scientific meetings

  1. Media coverage:
    If it bleeds, it leads— or a variation on that theme.
    If the ‘facts’ presented at a scientific conference seem too hard for the ‘average reader’ then dumb it down a little and leave out some of the longer words, like hypothetical or implausible.
    Most journalists write to a 5th to 8th grade level for the general public; I find that insulting but hey…….

    From Wiki, which is where I go first for dumbed down information (grin) but it’s a good jumping off place—–
    ****write as simply as possible and minimize the complexity of argument involved in a given written piece, often at the expense of factual accuracy, completeness, depth, and/or logical validity****

    Some of the reasons media reporters go to a scientific conference rather than read a dense science article (if it were even published yet) are:
    1. They might get to travel to a city they’ve always wanted to visit, expenses paid.
    2. There might be rooms with free flowing liquor available for the press and other conference participants.
    3. There might be a chance to ‘hook up’ with a lovely young lady (or hunk) there.

    There are more. 🙂

    • Hello Cave – I can’t speak for why journalists attend these conferences (most, I’m guessing, are simply assigned to cover the events by their editors) – my concern is not so much with the reporters themselves, but with ‘scientists’ presenting their papers as if they were gospel.

What do you think?

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s