This week, I happened upon an essay in Photon In The Darkness called A Layperson’s Guide to the Scientific Literature. While much of this site looks at the subject of autism and the growth of sensational theories based on individual experiences rather than medical research, there is much here about pseudoscience that certainly helps me, a dull-witted heart attack survivor, understand more about how to interpret ‘research’ that scientists (or their Big Pharma pals, whichever the case may be) publish in medical journals.
For example, this Layperson’s Guide discusses a phenomenon called The Lucky Stockbroker Syndrome that can lead to reinforcement of individual experiences, based on the following analogy:
“A stockbroker decides to try a new money-making scheme. He gets a list of 10,000 potential clients and sends half of them a letter telling them that a certain stock will rise in value over the next week. To the other half of the list, he sends a letter saying that the same stock will fall in value.
“The next week, he sends letters to the half of his list (5,000 people) that got the correct prediction – again, half of them are told that a certain stock will go up in price and the other half are told that the stock price will fall. This goes on for a total of six weeks.
“At the end of this time, he has a list of 156 people who have – by random chance – received six consecutive correct predictions about stock prices. He then offers these people an expensive five-year subscription to his stock-picking service, which they gladly purchase, thinking that he has some amazing system for predicting the stock market.”
The essay goes on to compare the Lucky Stockbroker with current autism therapies.
Parents who try a particular therapy for their children (such as chelation or dolphin therapy) prior to the child coincidentally having a period of improvement will conclude – erroneously – that the therapy caused the improvement. Those who try the same therapy and don’t see any improvement generally “move on” and try something else.
“Even worse, some practitioners exhort the parents to stay the course, ‘give it a chance to work’ or ‘don’t leave before the miracle’, which leads them to keep trying the therapy until – again, by sheer coincidence – the child goes into a period of improvement.
“Thus, the pattern-seeking center of their brains sees – erroneously – a ‘pattern’ of the therapy causing improvement. In fact, they are simply fooling themselves into seeing a pattern that isn’t there.
“This isn’t stupidity, gullibility or a lack of intelligence. It is simply how human brains work. This is why science is so persnickety about placebo controls, blinding of observers and randomizing subjects. We do it to keep from fooling ourselves, which we are all too capable of doing, given the chance.”
Another example of that “pattern-seeking center” that will seem familiar to my doctor and nurse friends working in hospital is explained like this:
“A doctor in the ER notices that there are runs of non-accidental trauma (medico-legalese for deliberate acts of violence) that coincide with the full moon. These could then be reported as being due to the full moon. However, the ER staff fail to notice that such runs occur at random – unrelated to the phase of the moon – due to random clustering. However, the runs of violence that don’t occur during the full moon are unconsciously discounted by the ‘pattern-seeking center’ of their brain.”
The Layperson’s Guide to the Scientific Literature goes on to cover the benefits and pitfalls of various kinds of research. Visit A Photon In The Darkness for these and other insightful takes on science.
For more detailed descriptions of a variety of research methodology, read Nagging 101.
Good stuff. Interesting perspective on the world of business and research.
At the same time, i find it distressing! Makes everything suspect.
Thanks for this …. I think!
This is true.
There are ways to “interpret” research results in such a way as to suit your purpose, a fact that has benefited the drug companies who can choose to suppress data not flattering to their products, or similarly adjust study methodology to be so vague as to be meaningless – until they want to make money. This throws all research into question, which is the real tragedy in all of this.
“….This throws all research into question…” I agree completely. I just returned from attending a very large conference where virtually every scientific paper presented had not yet been published, nor been peer-reviewed, just another self-absorbed opportunity to apply for more grant money to do more research, I deduced….
Love the lucky stockbroker example here. Brilliant.
Good work here. The Lucky Stockbroker is a great illustration that applies to many other “proven facts” – it shows how easy it can actually be to impress others with what appears to be solid “science”, but isn’t.
The Lucky Stockbroker Syndrome – brilliant! Thx for this link to Photon in the Darkness. Very enlightening for those of us non-researchers trying to make sense of medical studies especially those funded by big pharma.