(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
I’m speaking (remotely) to a roomful of accountants tomorrow. Exciting, huh? Actually, I don’t know if they’re accountants. They’re “accounting researchers,” whatever that means. . . .
The title they gave to my talk is “A statistician’s thoughts on registered reports.” There’s no abstract (and, of course, no slides) but they sent me this list of relevant issues:
1. Power is essential for any good empirical research. How effective is registration at improving power? The positive, given they way we are doing it, is that people can commit to gathering a lot of high-quality data, knowing that they will be published regardless of whether they support their results.
2. How helpful is it really to reduce HARKing [hypothesizing after results are known]?
3. How effective is registration in reducing p-hacking, especially given that so many of the interesting results in these papers are in supplementary analyses?
4. If you don’t like p-values and NHST [null hypothesis significance testing], is registration a step in the right or wrong direction. The way we’ve done it, everyone is spelling out hypotheses that are the focus of the paper. Is there a form of registration that gets away from this?
5. Where might Bayesian stats, thorough description, etc., fit in? Here’s one paper that explicitly incorporates Bayesian analysis . . . one of the very few in the field ever to do so: “Investor Behavior and the Benefits of Dispersed Stock Ownership” by Darren Bernard, Nicole Cade, and Frank Hodge.
6. We are pretty traditional NHST/frequentist. If you talk about alternatives to this, it will be the first time for many in this audience to think about them.
I think of accountants as pretty concerned with getting things right, and not over-claiming. So I can see how null hypothesis significance testing can be attractive to them (all that concern about keeping the error rate low, if the null hypothesis is actually true) but how it can also be a real trap, by pushing people toward that horrible alchemy where any statistical study, however noisy, leads to a deterministic statement about the effect being there or not. So it’s a real concern.
One thing I don’t really know is what accounting researchers do. Are they basically applied economists, writing econ papers in a particular area of application? Do they study the practices of real-world accountants? Do they try to come up with better ways of detecting fraud, etc? I guess they do all these things, along with others of which I’m not yet aware.
P.S. More here.
The post Can you account for this? appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
