Meta-analysis of PK Evidence - Dice-throwing. Parapsychologists Dean Radin and Diane Ferrari (1991) at Princeton University report the results of a meta-analysis of all dice-throwing experiments conducted between 1935-1987. Each experiment was categorized on the basis of a set of “quality criteria” that, according to Radin (1997, p. 311) cover virtually all often-cited design criticisms of psychokinetic (PK) experiments.
Automatic recording. Die faces were automatically recorded onto a permanent medium (e.g., photographed onto film).
Independent recording. Someone other than the experimenter also recorded the data.
Data selection prevented. Use of a data recording technique that ensured that all data were used in the final analysis (e.g., sequential-frame photographic data recordings, use of bound record books).
Data double-checked. Data were manually or automatically double-checked for accuracy.
Witnesses present. Witnesses were present during data recording to help reduce the possibility of mistakes or fraud.
Control noted. A control study was mentioned, but no details were published.
Local control. Control data were obtained under the same conditions as the experiment, using the same subject(s) in the same conditions, but with no specific mental effort applied to the dice.
Protocol control. The study was designed in such a manner that controls were inherently a part of the experiment (e.g., equal number of throws for each die face).
Calibration control. A long-term randomness test was conducted, usually immediately before and immediately after an experimental series..
Fixed run lengths. Optional stopping was ruled out by a prespecified design.
Formal study. The study used a prespecified methodology, as well as statistical analyses specified in advance of experimentation.
|
Odds against chance of more than a billion to one. Radin and Ferrari (1991) identified 148 experiments conducted by 52 experimenters using 2,569 participants who tried to mentally influence 2.6 million dice throws (compared to 31 control studies where no one attempted to influence 150,000 dice throws). Radin (1997) graphically summarized the results of the study by year with confidence intervals. While probability would predict a 50% hit rate simply by chance,
The overall hit rate for all control studies (i.e., studies in which no one tried to influence the tossed dice) was 50.02 percent, and the confidence interval was well within chance expectation, resulting in overall odds against chance of two to one. But for all experimental studies, the overall hit rate was 51.2 percent. This does not look like much, but statistically it results in odds against chance of more than a billion to one. (Radin, 1997, p. 134)
No replication problem, no inadequate controls, no file drawer problem. Radin & Ferrari’s (1991) meta-analysis of the overall combined experimental results also indicated that the overall hit rate was not due either to a few extremely successful studies or to a few exceptional investigators who reported the bulk of the successful experiments after these experiments/ investigators were excluded from the analysis. They calculated that an additional 17,974 unpublished, unsuccessful, unretrieved studies would have been needed to nullify the observed effect and reduce the overall odds down to p=.05 (i.e., 121 additional nonsignificant studies would have been required for each study included in the analysis, sometimes called the “file drawer” problem).
That many studies would have required each of the fifty-two investigators involved in these experiments to have conducted one unpublished, nonsignificant study per month, every month, for twenty-eight years. This isn’t a reasonable assumption; thus, selective reporting cannot explain these results. (Radin, 1997, pp. 135-136)
|
