Friday, April 3, 2009

Homeopathy: the p-Value Argument

In order to reject the null hypothesis in a given experiment, the results must deviate significantly from what we'd expect to see were the null true. In medicine and the social sciences, significance is defined mathematically as 1/20, meaning that if the null hypothesis is true, we'd make the mistake of rejecting it once every twenty runs of the experiment. The value of 1/20 or .05 is know as the "p-value".

In physics, significance is defined as 1/10,000. The bar is set higher because humans know more about the basic properties of matter than they do about pathophysiology. Laws of thermodynamics, electromagnetism, mechanics --all have been validated countless times by experiments over many decades. Thus physicists understand and can control for more of the relevant variables in their studies than doctors can.

Homeopathy is a set of claims about curing disease. "Like cures like," or the law of similars, dictates that a small amount of a substance that provokes the same symptoms as those presenting in the patient will cure the problem. This is a medical claim which might be tested experimentally like any other medical claim, using the 1/20 significance level.

But there are other homeopathic claims with greater implications for science generally. "The law of infinitesimals," or the homeopathic principle that the potency of a remedy increases with its dilution might be a medical claim, were the active ingredient within the dilution measurable. However, most homeopathic remedies are so dilute that no active ingredient remains present.

Given the above, one might think that tap water would work equally as well as homeopathy. However, the homeopaths claim that their remedies somehow remember or retain the "spirit-like" essence of the active component even when its every molecule has been diluted away.

Now, if we discover solid evidence in support of this notion of water-memory, doctors won't be the only ones busy re-writing textbooks. The physicists and the chemists will be equally as busy revising their explanatory models of the world. For the water-memory hypothesis isn't pertinent to medicine specifically. Rather, it's a claim concerning the fundamental properties of matter generally. Therefore, it's actually a physics claim.

For this reason, the p-value associated with any experiment designed to test homeopathy must be set at 1/10000 at minimum for even the most preliminary and inconclusive investigation. The 1/10000 is the standard we use for new results in the basic sciences that merely add to present understanding without contradicting established principles. The standard really ought to be higher for water-memory, as the notion violates basic laws of thermodynamics.

An excerpt from the NCCAM web site:
10. Is NCCAM funding research on homeopathy?
Yes, NCCAM supports a number of studies in this area. For example:
  • Homeopathy for physical, mental, and emotional symptoms of fibromyalgia (a chronic disorder involving widespread musculoskeletal pain, multiple tender points on the body, and fatigue).
  • Homeopathy for brain deterioration and damage in animal models for stroke and dementia.
  • The homeopathic remedy cadmium, to find out whether it can prevent damage to the cells of the prostate when those cells are exposed to toxins.
Although I've not reviewed the experimental designs of the above studies, it's a safe bet the significance level is set at 1/20 rather than the more appropriate 1/10000. If such is the case, I must call on the NIH to shut down those investigations immediately. Precious research resources should not be wasted on badly designed studies at the taxpayer's expense.


  1. Intrigueing, I hadn't heard the P value argument before.
    I don't know that it's entirely helpful though - homeopathy's claims aren't verifiable at the 1/20 P value as it is.
    Haven't they moved on from classical water memory to quantum woo?

  2. LOL. The homeopaths occasionally reach marginal significance at 1/20. And they're still trying, thanks to our tax dollars.

    I think if it were clear that they need results beyond a 1/10000 significance level, they'd be forced to throw in the towel.

  3. What a fascinating argument: homeopathy is a physics problem.

    Also, I have always understood 1/20 significance to be nothing special. It is simply an indication that you don't necessarily throw the data away. To be sure you have an effect, it is usual to require p=0.001, or 1/1000 chance of the effect being random. Funding something at the 1/20 level means you are going to waste a lot of money. Which I guess is what happens.

  4. Dr. B,

    Please see my comment on SBM:

  5. Hey Dr. A,

    I'm working my way through your links. Unfortunately, SBM has been crashing a lot over the past 24 hours. And today I'm booked to the teeth.

    I think the p-value argument has one advantage: it can be stated in a way that retarded people, middling bureaucrats, and even doctors can understand. "Bayesian" anything frightens them.

    Here's the middling bureaucrat version: "If it's got any physics in it, it's NSF not NIH. Thanks for your call. Buh-bye."