UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)
UvA-DARE (Digital Academic Repository)
Bayesian model selection with applications in social science
Wetzels, R.M.
Publication date
2012
Link to publication
Citation for published version (APA):
Wetzels, R. M. (2012). Bayesian model selection with applications in social science.
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.
Contents
Contents i
1 Introduction 1
1.1 Hypothesis Testing, an Example: The t Test . . . 1
1.2 Various Measures of Evidence . . . 2
1.3 The Bayes Factor . . . 3
1.4 Outline . . . 4
I
Bayesian Model Selection: Theoretical
9
2 How to Quantify Support For and Against the Null Hypothesis: A Flexible WinBUGS Implementation of a Default Bayesian t test 11 2.1 Introduction . . . 122.2 Bayesian Hypothesis Testing . . . 13
2.3 SD: An MCMC Sampling Based t Test . . . 15
2.4 The One-Sample SD t Test: Comparison to Rouder et al. . . 17
2.5 The Two-Sample SD t Test: Comparison to Rouder et al. . . 18
2.6 Extension 1: Order-Restrictions . . . 19
2.7 Extension 2: Variances Free to Vary in the Two-Sample t Test . . . 20
2.8 Summary and Conclusion . . . 24
3 An Encompassing Prior Generalization of the Savage-Dickey Density Ratio 27 3.1 Introduction . . . 28
3.2 Bayes Factors from the Encompassing Prior Approach . . . 28
3.3 The Borel-Kolmogorov Paradox . . . 33
3.4 Concluding Remarks . . . 37
4 A Default Bayesian Hypothesis Test for Correlations and Partial Cor-relations 39 4.1 Introduction . . . 40
4.2 Frequentist Test for the Presence of Correlation . . . 41
4.3 Frequentist Test for the Presence of Partial Correlation . . . 42
4.4 Bayesian Hypothesis Testing . . . 43
4.5 Default Prior Distributions for the Linear Model . . . 44
4.6 The JZS Bayes Factor for Correlation and Partial Correlation . . . 47
4.7 Concluding Remarks . . . 49
5 A Default Bayesian Hypothesis Test for ANOVA Designs 51 5.1 Introduction . . . 52
5.2 Bayesian Inference . . . 52
5.3 Linear Regression, ANOVA, and the Specification of g-Priors . . . 54
5.4 A Bayesian One-Way ANOVA . . . 57
5.5 A Bayesian Two-Way ANOVA . . . 60
Contents
5.6 Conclusion . . . 62
II Bayesian Model Selection: Applied
65
6 Statistical Evidence in Experimental Psychology: An Empirical Com-parison Using 855 t Tests 67 6.1 Introduction . . . 686.2 Three Measures of Evidence . . . 69
6.3 Comparing p Values, Effect Sizes and Bayes Factors . . . 73
6.4 Conclusions . . . 75
7 Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi 79 7.1 Introduction . . . 80
7.2 Problem 1: Exploration Instead of Confirmation . . . 81
7.3 Problem 2: Fallacy of the Transposed Conditional . . . 82
7.4 Problem 3: p values Overstate the Evidence Against the Null . . . 84
7.5 Guidelines for Confirmatory Research . . . 87
7.6 Concluding Comment . . . 89
8 An Agenda for Purely Confirmatory Research 91 8.1 Bad Science . . . 93
8.2 Good Science . . . 96
8.3 Example: Precognitive Detection of Erotic Stimuli? . . . 99
9 Discussion 103 9.1 Discussion . . . 103
9.2 Future Directions . . . 105
III Appendices
109
A Bayesian Parameter Estimation in the Expectancy Valence Model of the Iowa Gambling Task 111 A.1 Part I: Explanation of the Iowa Gambling Task and the Expectancy Va-lence Model . . . 113A.2 Part II: Maximum Likelihood Estimation . . . 115
A.3 Part III Bayesian Estimation . . . 120
A.4 Part IV Application to Experimental Data . . . 127
A.5 General Discussion . . . 133
B Bayesian Inference Using WBDev: A Tutorial for Social Scientists 135 B.1 Introduction . . . 136
B.2 Installing WBDev (BlackBox) . . . 137
B.3 Functions . . . 138
B.4 Distributions . . . 148
B.5 Discussion . . . 156 C Appendix to Chapter 4: “Calculating the Bayes Factor Using R” 159 D Appendix to Chapter 5: “Calculating the Bayes Factor Using R” 161
Contents
E Appendix to Chapter 7: “Bem: a Robustness Analysis” 163 F Appendix to Chapter 8: “Results from a Confirmatory Replication
Study of Bem (2011)” 167
F.1 Introduction . . . 167 F.2 Results From a Confirmatory Study . . . 168 F.3 Conclusion . . . 172
References 173
Nederlandse Samenvatting 187
Dankwoord 191
