Reproducibility and Replicability in Science (2019) / Chapter Skim
Currently Skimming:
Appendix D: Using Bayes Analysis for Hypothesis Testing
Appendix D: Using Bayes Analysis for Hypothesis Testing
Pages 221-230
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 221... ...
These odds or, equivalently, the probability, can be obtained from the Bayes formula. For purposes of exposition, it is convenient to express the Bayes formula using likelihood ratios in the simplified context of observing a data point xo and using it to test null hypothesis H0 versus the alternative hypothesis H1, as shown in Equation D.1.
|
From page 222... ...
In words, the Bayes formula (see Equation D.1) shows that the postexperimental odds favoring a hypothesis depends on the pre-experimental odds favoring the hypothesis and the relative likelihood of observing the results when the hypothesis is true, in comparison to the relative likelihood when the hypothesis is false.
|
From page 223... ...
Put another way, if the observed results happened to coincide with the mean value of the previously chosen alternative hypothesis, one would obtain the maximum possible change in the a posteriori (post-experiment) probability of the experimental hypothesis in comparison with the null hypothesis.
|
From page 224... ...
In principle, one would want to specify the prior odds without knowing the specific results of the study, based only on knowledge obtained prior to the study. One is expressing the odds as favoring the experimental or alternative hypothesis, but one could equivalently use the same results to estimate the post-experimental odds favoring the null hypothesis based on preexperimental odds of P [ H0 ]
|
From page 225... ...
Comparisons across levels of significance show the degree to which more statistically significant results affect the probability that the experimental hypothesis is correct: see Tables D-1, D-2, D-3, and D-4. However, the effect of the observed level of statistical significance is indirect, affected by sample size and variance, and mediated by the Bayes factor and the prior probabilities of the null and experimental hypotheses.
|
From page 226... ...
≈ 3.87. 1.6452 /2 Bayes factor (simplified calculation)
|
From page 227... ...
≈ 14.9. 2.3252 /2 Bayes factor (simplified calculation)
|
From page 228... ...
Without losing sight of the importance of errors in experimental design and execution or instances of fraud as sources of non-replicability, this excursion into Bayesian reasoning demonstrates how non-replicability can reflect the probabilistic nature of scientific research and be an integral part of progress in science. Just as it would be wrong to assume that any particular instance of non-replicability indicates a fundamental problem with that study or with a whole branch of science, it is equally wrong to ignore sources of non-replicability that are avoidable and the result of error or malfeasance.
|
From page 229... ...
. The Prior Odds of Testing a True Effect in Cognitive and Social Psychology.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.