Standards of evidence

From WikiSynergy

The crux of every issue with claims of the paranormal and the arguments of skeptics comes down to evidence. Proponents of paranormal phenomenon claim that there is evidence, while skeptics often claim that there is insufficient evidence.

One of the problems is that there is sometimes a gap between what each side considers sufficient and reliable evidence. This article outlines what constitutes scientific evidence. Such evidence standards are the same for all sciences, even if the way in which evidence is gathered necessarily differs between fields. Frontier topics are no different from mainstream topics in the standards of evidence necessary to establish them as well-grounded scientific fact.

Contents 1 Common evidence standards 1.1 Reliable and repeatable 1.2 Capable of being studied in controlled settings 1.3 Empirical and observation driven 1.4 Expert testimony should come from consensus 1.5 Acceptance of an hypothesis should flow from unique predictions 1.6 Evidence reaches statistical significance 1.7 Evidence in United States law 2 Considering claims in frontier subjects 2.1 Anecdotal claim 2.2 Empirically supported claims 2.3 When evidence is evidential 2.4 Subjective evidence 2.5 Objective evidence 2.6 Modern verses old evidence 3 References

Common evidence standards

Reliable and repeatable

The primary standard beyond anything else is that the conditions under which evidence is accumulated must be reliable and repeatable. If these conditions are not met then there is no way of analyzing the situation or in assessing the accuracy of the claims or the evidence.

Capable of being studied in controlled settings

While not as important as reliability and repeatability, controllablity is also very important. A controlled study helps limit extraneous variables that complicate and blur the validity of evidence. The key to a well controlled experiment is that you are testing only the specific claim being made and not a million other things.

While not always possible, the real gold standard for evidence is from double-blind or triple-blind studies. These are cases were neither the experimenter or subject are aware of whether they are in a control group or an experimental group. This kind of evidence is best for claims such as psychic healing, ESP, hypnotism, or homeopathy, and may also be used to study other paranormal claims.

However, as Michael Shermer writes,

not all claims are subject to laboratory experiments and statistical tests. Many historical and inferential sciences require nuanced analyses of data and a convergence of evidence from multiple lines of inquiry that point to an unmistakable conclusion. Just as detectives employ the convergence of evidence technique to deduce who most likely committed a crime, scientists employ the method to determine the likeliest explanation for a particular phenomenon.

Once an inferential or historical science is well established through the accumulation of positive evidence, however, it is just as sound as a laboratory or experimental science.^[1]

See also Pseudoskepticism#13

Empirical and observation driven

Evidence should ideally be derived from first-hand empirical observation, which has been reliably documented. It is far better to observe an event first-hand than to hear about it from someone else, who heard about it from someone who really saw it.

Expert testimony should come from consensus

No one can be an expert in everything. The amount of information on even the most specific of topics can easily overwhelm the novice. Both sides must rely on expert opinion. Expert opinion is derived from a consensus of individuals with relevant education and expertise in the topic. Expert opinion does not come from only one individual, and it does not come from individuals with only marginal training on the subject area or whose training is outside the subject area.

See also Scientific consensus and frontier subject areas and Pseudoskepticism#Inexpert opinion.

Acceptance of an hypothesis should flow from unique predictions

If evidence is being proposed to support one hypothesis over another, that evidence should stem from a unique prediction of one hypothesis. If both hypotheses equally predict the same result then the data is not evidence for either hypothesis (see also Occam's razor).

Evidence reaches statistical significance

Statistics is a way of analyzing the probability of a data set to determine if it is significant enough to provide sufficient evidence for a claim. Evidence should therefore have enough "meat" to it, with enough observations, and a big enough difference between control and experimental groups, to reach statistical significance. A single observation does not evidence make.

Evidence in United States law

Courts have taken steps to make sure that only true expertise is allowed in a courtroom. Allowing "expert testimony" in court on matters of science, medicine, or other technical subjects has an extremely prejudicial effect, and tends to persuade a naive jury more by authority than by actual reasoning. In short, junk science may hide behind fancy degrees. To avoid this prejudicial effect, courts have recently begun to hold gatekeeping hearings to prevent the introduction of junk science.^[2] The court's gatekeeping duty, inferred from Federal Rule of Evidence 702, "imposes a special obligation upon a trial judge to 'ensure that any and all scientific testimony . . . is not only relevant, but reliable.'"^[3] If expert testimony is not reliable, it will run the risk of prejudicing the jury one way or another; thus, the expert testimony should not only be discredited and ignored by the jury, but it should not even be presented to the jury.

Daubert and its progeny lay out factors to consider when deciding whether or not evidence is reliable. Courts use a four-factor test, asking:

1. Whether a theory or technique can or has been tested,
2. Whether the expert's testimony has passed through the processes of peer review and publication,
3. Whether there is a high error rate in a theory or technique, and whether standards for evaluating such errors exist, and, most importantly,
4. "Whether the theory or technique enjoys 'general acceptance' within a 'relevant scientific community.'"^[4]

Thus, expert testimony (scientific or otherwise) must pass a rigorous test to even be admitted at trial. An expert's ipse dixit statements - i.e., "I am an expert, and I say this, therefore it must be true" - are not admissible of their own accord.^[5] In short, experts should use in the courtroom the same degree of logical scrutiny demanded by their field.^[6] Every step of the expert's analysis is subject to this exacting inquiry; no logical leaps are allowed, and the court should be able to follow an expert's reasoning from start to finish.^[7] Further, where an expert draws support for a statement from a published study, or other corpus of knowledge, the statement cannot run counter to the study: swearing one way based on a study that suggests the other is not allowed.^[8]

Zealous guardianship therefore characterizes the manner in which American courts should treat scientific evidence. The highest scientific standards are applied to any testimony by a so-called expert; no arguments from authority are allowed; and peer review, along with general scientific acceptance, is a must. (from RationalWiki [1])

Considering claims in frontier subjects

A paranormal claim will be considered based on the amount and kind of supporting data. For instance:

Anecdotal claim

An observation made in the field under uncontrolled conditions will generally be considered an unsubstantiated and unsubstantiable event unless the claim is supported by objective evidence. The question of what evidence is evidential is discussed below.

Empirically supported claims

The product of controlled research, following a well-supported protocol, is considered evidential if the results are vetted by qualified peers. However, a single such report would not be considered evidential by itself. It would be a data point to be considered along with similar experiments in meta analysis to produce a base of evidence which can be evidential.

When evidence is evidential

Evidence is in the form of subjective and objective. It is also historical and modern, documented by a trained observer or by a "witness."

Subjective evidence

When a person reports seeing a ghost, the first question must concern the reliability of the person (the witness). It is understood that the human powers of observation are very unreliable, and that what is experienced is colored by expectations, worldview and circumstance. A door caused to close by a naturally occurring draft in the room might be reported as a door being closed by a draft by a pragmatic scientist, a ghost closing the door by an excitable ghost hunter or a door that moved, and needs to be examined by a hauntings investigator.

As a rule, reports made by witnesses are not evidential and are realistically only useful to help researchers understand what occurred so as to design hypotheses and experiments to determine how and why the event occurred.

Objective evidence

Physical evidence that an event occurred is evidential if the chain of custody is understood. The problem is that technology allows audio and visual forms of evidence to be simulated. It is possible "photo shop" a photograph to make it appear paranormal phenomena was captured. This means the reliability of the photographer is part of the evidence.

Reflective phenomena, such as faces in light reflected from windows may be evidential if visual instrumental transcommunication is considered, but as spirit photography, it is useless if the scene being reflected is not known. Here, possible communication by the paranormal transforming of reflected light to produce an image is very different from capturing the face of a ghost that happened to be in the scene. Evidence for one is not the same as evidence for the other. See the explanation here.

Modern verses old evidence

As evidence, anything older than ten years should be suspect because of the change in technology and improved understanding of human nature.

References

1. ↑ Michael Shermer I Want to Believe
2. ↑ Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993) (defining the court's gatekeeping function).
3. ↑ Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147 (U.S. 1999) (citing Daubert, 509 U.S. at 586).
4. ↑ Kumho, 526 U.S. at 150 (citing Daubert, 509 U.S. at 592-594).
5. ↑ Kumho, 526 U.S. at 157.
6. ↑ Vargas v. Lee, 317 F.3d 498, 500 (5th Cir. 2003).
7. ↑ Knight v. Kirby Inland Marine, 482 F.3d 347, 355 (5th Cir. 2007).
8. ↑ Doe v. Dallas Ind. Sch. Dist., 220 F.3d 380, 386 (5th Cir. 2000).