Blog
A Methodological Metamorphosis: Bayesian Inference and Open Science in Psychology
Last week I presented at the APS Annual Convention in Barcelona, as part of the symposium "The Past, Present, and Future of Scientific Reform." The talk summarized a large collaborative project tracking the adoption of Bayesian inference, preregistration, and open data across 28,745 empirical articles from 12 psychology journals spanning 2004–2024 — the work of 44 co-authors and one LLM.
In the top-tier journals (e.g., Psychological Science, Journal of Experimental Psychology: General), by 2024 about 75% of articles shared open data, 40% were preregistered, and 25% used Bayesian inference. All three practices were essentially absent before 2010 and took off sharply after 2015.
A key addition to the project — prompted by peer review — was a second sample of six mid-tier journals (including Psychological Reports, International Journal of Psychology, and Journal of Applied Social Psychology), covering another 13,111 articles coded entirely by LLM. The contrast is striking: open data sits around 40%, preregistration around 15%, and Bayesian inference around 15% in mid-tier journals — roughly half the rates seen in top-tier outlets. Most mid-tier journals remain below 25% on all three practices.
The most interesting exception is Memory & Cognition, which matches top-tier journals on open data (~80%) and Bayesian adoption (~30%), despite its mid-tier classification. That outlier points to journal policy as a stronger driver of reform adoption than prestige alone.
Psychology is on a genuinely promising path — the metamorphosis is real — but the gap between journal tiers makes clear that open science practices have not diffused evenly across the field. Preprint: https://doi.org/10.31234/osf.io/ck3js_v1.
In the top-tier journals (e.g., Psychological Science, Journal of Experimental Psychology: General), by 2024 about 75% of articles shared open data, 40% were preregistered, and 25% used Bayesian inference. All three practices were essentially absent before 2010 and took off sharply after 2015.
A key addition to the project — prompted by peer review — was a second sample of six mid-tier journals (including Psychological Reports, International Journal of Psychology, and Journal of Applied Social Psychology), covering another 13,111 articles coded entirely by LLM. The contrast is striking: open data sits around 40%, preregistration around 15%, and Bayesian inference around 15% in mid-tier journals — roughly half the rates seen in top-tier outlets. Most mid-tier journals remain below 25% on all three practices.
The most interesting exception is Memory & Cognition, which matches top-tier journals on open data (~80%) and Bayesian adoption (~30%), despite its mid-tier classification. That outlier points to journal policy as a stronger driver of reform adoption than prestige alone.
Psychology is on a genuinely promising path — the metamorphosis is real — but the gap between journal tiers makes clear that open science practices have not diffused evenly across the field. Preprint: https://doi.org/10.31234/osf.io/ck3js_v1.
Bayes Factors for Structural Equation Models with Bridge Sampling and blavaan
I recently completed a very big chunk of my WB fellowship project. The output: A new paper on Bayesian model comparison in structural equation modeling (SEM). The central question is simple: how can we compute Bayes factors for SEMs in a way that is practical, transparent, and flexible enough to incorporate substantive prior information?
For the curious: The preprint is available at https://doi.org/10.31234/osf.io/pt2bc_v1.