Using a new method they developed for analyzing social-science data, a pair of sociologists was “surprised” to discover that, no matter how they sliced the data, they could not refute a 2012 study finding that children of same-sex “parents” fare much worse than those of married heterosexual parents.
In March, Cambridge University Press published Multiverse Analysis, a book by Cornell University sociologists Cristobal Young and Erin Cumberworth. The publisher’s description:
There are many ways of conducting an analysis, but most studies show only a few carefully curated estimates. Applied research involves a complex array of analytical decisions, often leading to a “garden of forking paths” where each choice can lead to different results. By systematically exploring how alternative analytical choices affect the findings, Multiverse Analysis reveals the full range of estimates that the data can support and uncovers insights that single-path analyses often miss.
Young and Cumberworth reviewed a number of disputed studies and found that most failed to withstand multiverse analysis. The studies were riddled with biased assumptions that, conveniently, produced exactly the results the researchers wanted. For example, a study that purported to show that women’s political views shifted with their ovulation cycles was so rigged that it “presented the most extreme estimate available from the multiverse of possible results,” the sociologists found.
The big test, though, was whether the 2012 study by University of Texas sociology professor Mark Regnerus would hold up under multiverse analysis. That study of about 3,000 young adults, 248 of whom had same-sex parents, touched off a firestorm — not because its methodology was invalid but because it came to the “wrong” conclusion.
Catholic University of America sociology professor Paul Sullins recounted:
Examining this powerful set of data, Regnerus reported “numerous, consistent differences” that disadvantaged same-sex-parented persons. Compared on forty outcomes to persons raised by stable biological parents, persons raised by lesbians differed on twenty-five of them (63 percent), those by gay males on eleven (28 percent). The lesbian-parented children suffered higher depression, lower physical health, and lower income and educational progress. They were also more often unemployed or on public assistance and more likely to have been arrested and to have pleaded guilty to a serious crime. They were much more likely to have been sexually abused as children, to have had an affair while married or cohabiting, and to report that their current relationship was in trouble.
Regnerus did not mince words about the implications of these findings, declaring: “the empirical claim that no notable differences exist must go.” Unlike [his study], he asserted, studies focusing on parenting ability or parent reports of current child well-being “will fail to reveal — because they have not measured it — how their children fare as adults.” Further: “The small or nonprobability samples so often relied upon in nearly all previous studies have very likely underestimated the number and magnitude of real differences.”
Of course, those previous studies were conducted in such unprofessional ways (for example, by asking parents in gay bookstores how their children were doing) precisely because they “proved” there was no difference between traditional and same-sex parenting.
Regnerus and his study were denounced left and right — or perhaps just left — in the media and academia. “Hundreds of scholars and activists — the distinction was often unclear — demanded retraction of the study and investigation of Regnerus for misconduct,” wrote Sullins. (Regnerus emerged victorious.) Others tried applying their own analyses to Regnerus’ data to correct the “flaws” they detected in it and generate the results they desired.
Young and Cumberworth were, therefore, shocked at what their analysis uncovered, penned Sullins:
The widespread critique of this highly disputed study resulted in a multiverse of more than two million alternative analyses that were statistically significant (meaning the results could not be the result of chance variation due to random sampling). Initially anticipating that “a comprehensive multiverse analysis would drive [the study’s many critics’] point home in a powerfully conclusive way,” Young and Cumberworth instead found something unexpected and remarkable: not one of the two million significant alternatives resulted in positive outcomes for LGBT-parented children. Although often with smaller effects, every analysis confirmed the Regnerus study’s central finding that children turned out better with intact biological parents than with LGBT parents. Regnerus’s thesis, it turns out, was not only true in the analytic model in which he presented it: it was true in every analytic model possible.
The authors confessed that they were “surprised by the robustness of the Regnerus finding.”
As to those reworkings of Regnerus’ data that purported to come to the opposite conclusion, Young and Cumberworth found that, along with making other dubious decisions, the researchers “stacked the deck … by analytical choices that reduced the sample size,” Sullins wrote. This enabled them to claim there was no statistically significant difference in outcomes for children in same-sex versus traditional families.
According to Sullins, Young and Cumberworth hope their findings will reopen the debate on Regnerus’ study and same-sex parenting:
By treating the negative “LGBT parent effect” as a hypothesis to be considered on the same grounds as any other hypothesis, and not as a malicious idea to be recoiled from, they hold out the hope of reclaiming open science and open debate on ideologically heterodox ideas in scholarly debate and in the public square.
Given the increasing intolerance of academia and the media, that may be asking too much.