(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
From a couple years ago:
In the context of a listserv discussion about replication in psychology experiments, someone wrote:
The current best estimate of the effect size is somewhere in between the original study and the replication’s reported value.
This conciliatory, split-the-difference statement sounds reasonable, and it might well represent good politics in the context of a war over replications—but from a statistical perspective I strongly disagree with it, for the following reason.
The original study’s estimate typically has a huge bias (due to the statistical significance filter). The estimate from the replicated study, assuming it’s a preregistered replication, is unbiased. I think in such a setting the safest course is to use the replication’s reported value as our current best estimate. That doesn’t mean that the original study is “wrong,” but it is wrong to report a biased estimate as if it’s unbiased.
And this doesn’t even bring in the possibility of an informative prior distribution, which in these sorts of examples could bring the estimate even closer to zero.
The post A completely reasonable-sounding statement with which I strongly disagree appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
