Extraordinary claims require extraordinary evidence

From WikiSynergy

In this famous statement by philosopher William James, spoken in the late 1800s, existence of a white crow is the extraordinary claim. This article explores the proper scientific protocol to use when exploring such claims.

Proper and improper use of the axiom

Skeptics often say that "Extraordinary claims require extraordinary evidence." For this to be valid argumentative tool, the skeptic must precisely define why a claim is extraordinary. If there is no precise definition of why a particular claim is extraordinary, there is no way to define exactly what evidence would suffice, and it therefore must be assumed that only normal levels of scientific evidence are required to prove the claim. Therefore when saying Extraordinary claims require extraordinary evidence a skeptic must define 1) how extraordinary the claim in question is, and 2) what evidence would meet the standard of "extraordinary." However, while there are some concrete ways of analyzing whether a claim is extraordinary or not, and what level of evidence should be required before the claim in accepted, there is no way to completely divorce the definition of "extraordinary" from the speaker's common sense and a-priori assumptions. Thus, what a person will accept as "Extraordinary evidence" always has something to do with their psychology. One way to define an "extraordinary" claim is through the use of Bayesian inference.

Bayesian inference

See also Bayesian probability

Bayes' equation is a way of relating the probability of evidence given a particular hypothesis to the probability of a hypothesis given the known data. There are two main pieces to the equation:

1. The likelihood function which is how likely evidence is if it is assume the hypothesis of interest is true.

2. The prior, which is the initial assumption about how probable the hypothesis is before the evidence is examined.

If the prior is low relative to other hypotheses which explain the same thing, then the evidence has to be a very good before the hypothesis should be accepted. If the prior for the new hypothesis is high or equal to that of other hypotheses then it requires less extraordinary evidence.

Priors can also be looked at retroactively. After examining how well a particular evidence set matches a given hypothesis, it is reasonable to ask "what would the initial beliefs about this hypothesis have to be for this evidence to be convincing?"

Assigning probability

Since there is no technical way of assigning the prior, it can only be set by common sense. It is thus open to prejudice and bias. Thus, it is often argued that skeptics use their own psychology as the arbiter of scientific validity when setting the prior low for frontier phenomena.

The claim that the skeptic's psychological state is used as the arbiter of scientific truth stems from the problem of quantifying prior probabilities if little or no evidence is available before an experiment. The skeptical argument is that a claim that goes against established scientific consensus, and violates known physical laws, should by its very nature be assigned a low "prior" probability of being true. When a claim contradicts prior knowledge, the evidence must be a very strong before we can be confident that the claim is true. This mathematical approach to probability analysis can then be summed up with the quip "extraordinary claims require extraordinary evidence." There are different levels of proof required for different kinds of claims:

1. A claim which contradicts prior scientific theory: Extremely high. Example: perpetual motion machines.
2. A claim which requires expansion of current scientific theory: Very high. Example: Telepathy, UFOs, and dark matter/energy.
3. A claim which, while unprecedented, requires no extension of current scientific theory: High. Example: Bigfoot.
4. A claim which extends current scientific knowledge: Ordinary evidence levels. Example: the discovery of a new species of bacteria.

Note the difference between the discovery of Bigfoot and a new bacterium. The difference is that while it is known that bacteria exist and that there are many undiscovered species, Bigfoot is a large creature which is evidenced only be anecdotal evidence. If Bigfoot existed it is strange that conclusive remains have not been discovered. Note that this example highlights the "common sense" part of setting the prior.^[2]

Assigning the prior for paranormal claims

There are few areas of frontier thought which can be said to actually controvert a natural law. One example would be perpetual motion machines, which violate the law of conservation of energy. Another might be homeopathy, which postulates an effect via unknown means. However, in nearly all cases, frontier claims do not violate any known natural laws; they simply have no known mechanism under known natural laws. They therefore may not require us to revise our understanding of natural law, but rather extend it. This means that, contrary to the claims of skeptics, most frontier claims do not need to have evidence which is extraordinary enough that we would feel comfortable throwing out current knowledge. (See also Discussion here).

Therefore, paranormal claims must have evidence extraordinary enough that we would be willing to say that we have detected a phenomenon which requires us to extend our current understanding of natural law. While this hurdle to acceptance is very high, it is not nearly as high as it would be for a phenomenon which actually violates known physical laws.

To take a "concrete" example, for phenomena such as telepathy, the inverse square law of radiation is apparently violated. That is, if telepathy is a radiation phenomenon, it violates the inverse square law (as well as the conservation of energy). That is apparently one reason why telepathy is proposed to be more like quantum entanglement, which is an observed phenomenon in which action at a distance occurs in the absence of radiation.

The person making the claim and presenting evidence is not obliged to explain why prior claims might be invalid. For instance, if the prior claim was based on science that ignored an aspect of reality, then it is up to those making the claim to adjust their theories to reflect reality.

An example: Natasha Demkina

According to Ray Hyman who evaluated Natasha Demkina

What is extraordinary

See also Considering claims in frontier subjects

Any claim: Any scientific claim must include qualifying statements. For instance, a claim that a ghost has been seen should be accompanied by an explanation of why what is being reported is a ghost and not a tourist who happened to be walking by.

Miracles: If an event is said to be a miracle, then it probably qualifies as an extraordinary claim. However, by labeling something a miracle, it become by definition, a matter of faith and no scientific claim can be associated with it. At the same time, no rebuttal is necessary except within the system of belief. Miracles are not a matter for scientific debate.

Paranormal: If an event is said to be paranormal (also, phenomenal), then it is by definition an extraordinary claim. By definition, the event falls outside of the norm, is not what is "normally" expected, and is therefore, not supported by current, established theory.

Scientific mistake: If a claim is made that at least part of a mainstream theory of science is incorrect, then the claim is not extraordinary in the sense intended by "paranormal," and is subject to the existing supporting data for that theory. Additional data may be required to make the point, but this is not a matter for frontier subjects.

See also

• Bayesian on Rationalwiki This article seems technical rather than the usual humor and derision.

References

1. ↑ Myers, Gerald E., William James: His Life and Thought, Yale University Press, 1986
2. ↑ Strictly speaking, all claims require exactly the same amount of evidence, it’s just that most "ordinary" claims are already backed by extraordinary evidence that you don’t think about. When we say “extraordinary claims”, what we actually mean are claims that do not already have evidence supporting them, or sometimes claims that have extraordinary evidence against them. Extraordinary claims require extraordinary evidence because they usually contradict claims that are backed by extraordinary evidence. The evidence for the extraordinary claim must support the new claim as well as explain why the old claims that are now being abandoned, previously appeared to be correct. The extraordinary evidence must account for the abandoned claim, while also explaining the new one.Skeptico
3. ↑ Statistics and the Test of Natasha By Ray Hyman