Why replication is difficult to achieve in psychological science. This does not mean that every experimental effect of psi can be easily produced on demand in the laboratory.
Few human capabilities are perfectly replicable on demand. For instance, even the best hitters in the major baseball leagues cannot hit on demand. Nor can we predict when someone will hit or when they will score a home run. In fact, we cannot even predict whether or not a home run will occur in a particular game. That does not mean that home runs don’t exist. (Utts, 2001, pp. 112-113)
Psi, like most human abilities, is variable. Like most complex and subtle psychological traits and states, psi skills and abilities are difficult to capture with traditional laboratory techniques. Most normal psychological effects are known to be exceptionally difficult to repeat (S. Epstein, 1980). Human behavior and cognition is highly variable, and experiments involving human beings never turn our exactly the same way twice.
Positive replications of previous studies in psychology is rare. Surprisingly, given the high value placed on the ability to repeatedly demonstrate the reality of claimed psychological effects commonly described in introductory psychology textbooks, positive replications of previous studies are exceedingly rare in psychology in particular, and in science generally (Bozarth and Roberts, 1972; Collins, 1985). Two likely reasons why replications are rarely conducted in behavioral science research is because of the time and expense involved in repeating well-designed, rigorous experiments, and because many scientific journals have editorial policies that discourage publication of “mere” replications (Neuliep & Crandall, 1991).
|
“The situation is not quite as dismal for psi research, [however], because psi is so curious and represents such a huge challenge to scientific assumptions, hundreds of investigators over the years have conducted thousands of replication studies” (Radin, 1997, p. 39).
Theory-Building Approaches: Meta-Analysis
How do we know that psi experiments have been replicated? We know that psi experiments have been replicated by measuring how much replication has taken place (Utts, 1991). The technique for assessing this is called “meta-analysis” or “the analysis of analysis” (see Broughton, 1991, pp. 279-284; Glass, McGaw, & Smith, 1981). The basis units of analysis in meta-analysis are the results of individual experiments, instead of the results of individual subjects. By combining thousands of people’s performances over hundreds of similar experiments, coding the experimental procedures used, quantifying effect sizes, calculating confidence intervals, and analyzing results to detect clear patterns across studies, highly reliable and accurate estimates about the repeatability of any highly variable phenomenon involving human performance can be obtained (Cohen, 1988; Cook, Cooper, Cordray, et al., 1992; Copper & Rosenthal, 1980; Glass, 1976; Hedges, 1987; Rosenthal, 1991). Meta-analyses have made a ground-breaking and persuasive case for the reality of psychic functioning (e.g., Bem & Honorton, 1994; Honorton, 1985; Honorton & Ferrari, 1989; Nelson & Radin, 2001; Jahn , Dunne, Nelson, et al., 2001).
|
