(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Don Green points us to this quantitative/qualitative meta-analysis he did with Betsy Levy Paluck and Seth Green. The paper begins:
This paper evaluates the state of contact hypothesis research from a policy perspective. Building on Pettigrew and Tropp’s (2006) influential meta-analysis, we assemble all intergroup contact studies that feature random assignment and delayed outcome measures, of which there are 27 in total. . . . We find the evidence . . . consistent with Pettigrew and Tropp’s (2006) conclusion that contact “typically reduces prejudice.” At the same time, our meta-analysis suggests that contact’s effects vary, with interventions directed at ethnic or racial prejudice generating substantially weaker effects. Moreover, our inventory of relevant studies reveals important gaps, most notably the absence of studies addressing adults’ racial or ethnic prejudice, an important limitation for both theory and policy. We also call attention to the lack of research that systematically investigates the scope conditions suggested by Allport (1954) under which contact is most influential.
I like that they don’t just give their conclusions; they also talk about the limitations of their data.
The contact hypothesis is a big deal, and I know that Don Green and his collaborators have been thinking a lot about meta-analysis in recent years, so I’m glad to see this research being done.
I’ve not yet read the article, but I did notice that it doesn’t have a lot of graphs—and those that it does have, are close to impossible to read. This just seems to be a preprint, though, so maybe someone can help them visualize their data and their findings, and some real graphs can be added to the paper before it is published in its final version. With all this work on data gathering and data analysis, it would be a pity for the results to not be explored. There should be lots more stuff to be uncovered from some graphical displays.
P.S. The second author of the paper, Seth Green, is a Columbia political science graduate and is currently working at a startup called “Code Ocean.” Googling led to this webpage which says nothing about Columbia. Apparently Code Ocean is “a cloud-based computational reproducibility platform” with a mission to “to make the world’s scientific code more reusable, executable and reproducible.”
It also says on that webpage that Code Ocean handles code in Python, R, and eight other programming languages. If so, I think they should handle Stan code too! In all seriousness, I think a bit of Stan would help this project, in that it would push the researchers toward modeling the effects of interest, and away from time-wasters such as this: “We begin our quantitative analysis by assessing cross-study heterogeneity using Cochran’s Q. The test decisively rejects the null hypothesis of homogeneity of effects across studies (Q(26) = 173.563, p < .001; I^2 = 0.85). We therefore reject the fixed-effects meta-analysis model in favor of a random-effects meta-analysis model, where the variance of the normal random component is estimated using method of moments. The resulting estimate is 0.39, with a 95% confidence interval ranging from 0.234 to 0.555.” Or this: “The results presented in Table 3 suggest that a one-unit increase in standard error is associated with a 2.07 unit increase in effect size, although the pattern is of borderline significance (two-tailed p-value = .049).” I know they’re just doing their best, and I like the general flow of this analysis; it would just be easier in Stan to get to the substance.
Traditionally, classical analyses based on statistical significance have been considered to be the safe option when analyzing data in the social sciences. But as we continue to work in areas where data are sparse and noisy, and effects are highly variable, it will make sense to just start with Bayes, to say goodbye forever to “p less than .05,” and to use Stan as your first rather than last resort for managing uncertainty and variation.
P.P.S. Cat picture demonstrating contact hypothesis from Diana Senechal.
The post The Other Side of the Night appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
