Let’s say you are a researcher hard at work analyzing results from your current medical project. You have a large number of available options that may affect the results of that research before coming to any published conclusions. (And if Big Pharma is funding your research, you likely have considerable motivation to skew those results to favour their product). What you will certainly be most interested in is simply interpreting your existing data by applying deductive reasoning to come up with “The Truth”. In other words, what are you going to tell the world that your findings mean? But according to the clever scientific minds over at Photon In The Darkness, deductive reasoning is not all it’s cracked up to be:

“Deductive reasoning is defined as ‘an argument where the conclusion is a logical consequence of its premise’. Many believe that if their conclusions follow logically from their premises, then their conclusions must be true. But this reasoning can be either valid (if the conclusions are logical consequences of their premises) or invalid (if they are not).

“There is no true.”

In an essay last year called Deductive Mis-Reasoning,  filed under their Help For The Bewildered section, Photon In The Darkness offers us the perfect simplistic example of flawed deductive reasoning at work: the iconic fictional police detective, Sherlock Holmes. Remember how author Sir Arthur Conan Doyle‘s Sherlock could pick up the most innocuous clue and then somehow solve the entire murder mystery based on his deductive reasoning about the clue?

Let’s compare the fictional deductive reasoning success of Sherlock Holmes with the equally fictional conclusions of some medical research.

Photon first examines this paragraph about a specific clue – the mystery man’s hat – in a Sherlock Holmes story, with examples of his deductive reasoning in square brackets:

“That the man was highly intellectual is of course obvious on the face of it [large hat size], and also that he was fairly well-to-do within the last three years [expensive hat], although he has now fallen upon evil days [wearing a hat three years out-of-style].

“He had foresight [had a hat-securer installed on the hat], but has less now than formerly [elastic of the hat-securer is broken and not replaced], pointing to a moral retrogression, which, when taken with the decline of his fortunes, seems to indicate some evil influence, probably drink, at work upon him.

“This may account for the obvious fact that his wife has ceased to love him [the hat was dusty – not brushed. Presence of a wife inferred because the man was carrying a goose.].”

These are all possible conclusions – thus, according to logic, valid conclusions – but not necessarily correct conclusions. Here are some alternative conclusions from Photon In The Darkness that appear equally valid and yet give a completely different picture of the hat’s owner.

[1] Large hat size: either a large man (big people have big heads) or a large head size due to untreated (as it would be in that time) childhood hydrocephalus. The first would give no information about intellect, while the second would be consistent with reduced intellectual capacity. Of course, we now know that head size is no predictor of intellectual ability.

[2] Expensive hat: could have been purchased second-hand (which would also explain much of the other premises). Might have been a hand-me-down from a wealthier relative or friend. Could have been stolen.

[3] Hat three years out-of-style: could have been purchased second-hand or given from a friend, relative, or generous stranger.

[4] Elastic hat-securer: see [2] and [3] above.

[5] Elastic of hat securer broken and not replaced: see [2] and [3] above.

[6] Dusty hat not brushed: too lazy to brush his own hat, slovenly, near-sighted and can’t see the dust. If we assume (as Holmes apparently did) that men of that time did not brush their own hats (even if they are noticeably dusty), he might have been a bachelor, widower or a transient visitor to London without his spouse. And even if we assume that men of that era did not cook geese, it does not mean that he was married. It could have been for a sister, the wife of a friend, his landlady, significant other, etc. who might cook a goose for him but not brush his hat.

So, with alternative conclusions that are consistent with the valid data at hand, Photon draws us a very different picture of the hat’s errant owner.

In fact, there are often several conclusions that could be drawn from any collection of data, but usually only one is correct.

Take for example the controversial medical research published in the summer of 2003 in the International Journal of Toxicology, headlined:

“Reduced levels of mercury in first baby haircuts of autistic children.”

The conclusion of the researchers was based on deductive reasoning. They had compared first baby haircut samples of 94 autistic children with a control group of 45 non-autistic adults – and there’s your first clue of flawed research methodology. Their conclusion:

“The lack of mercury in the hair of autistics may be due to a decrease in blood mercury levels feeding the hair follicles. This decrease is likely caused by the retention of the mercury inside the cells where it most likely causes its major biological damage. 

“Despite hair levels suggesting low exposure, these infants had measured exposures at least equal to a control population, suggesting that control infants were able to eliminate mercury more effectively.”

However, about a year later, a larger and more comprehensive study on the hair mercury levels of over 800 children ages 1-5 (which includes that “first baby haircut”) was published in the journal Environmental Health Perspectives. This much more extensive study essentially wiped out the first premise of the flawed 2003 conclusion.

As Photon In The Darkness observes:

“Even if we grant them their obviously flawed premise (in which the 2003 control group had hair mercury levels which averaged 16 times the mean hair mercury of the 800+ children in the 2004 survey), there are some alternative conclusions that do not require inventing reasons.”

For example:

• 1.  The autistic children had been exposed to less mercury than the control group.
• 2.  The higher mercury exposure of the control group actually prevented autism.
• 3.  Why did the control group have hair mercury levels far out of the accepted “normal” range? Two possibilities: either the control group had a massive and undiscovered exposure to mercury in their early years, or the hair mercury analysis of this group had been botched in the lab.

Photon observed that not only did the 2003 authors fail to consider most of the alternative conclusions that were consistent with their premises, it turns out that one of their premises – the major one – was clearly incorrect.

Read the entire article from Photon In The Darkness.

And for more basic information about scientific research, visit Nagging 101.

See also:

• Dr. Harriet Hall Explains Tooth Fairy Science
• Warning: Clinical Trials Funded By Drug Companies May Appear More Truthful Than They Actually Are

.