Why So Many Social Science Claims Are False
One of the most urgent questions facing society today is the potential impact of AI on education and labor markets. So it’s no surprise that two scientists attempted to provide an answer, producing a meta-analysis suggesting that incorporating ChatGPT in schooling dramatically improved student outcomes. Social media and online sources promoted the 2025 study as a beacon of bright light.
The problem is that the researchers’ conclusions were later found to be unreliable, and the study was retracted in April – another blow to the public’s faith in science.
Scientists are increasingly chasing answers to big questions in fields from health care to climate change to AI. The academic research industry incentivizes them to produce the biggest possible findings that make for splashy headlines and career promotions. But such discoveries are rare in science, and too often, the result is widespread exaggeration, dodgy methods, and miscommunication.
“The incentive structure of modern science is such that a ‘simplify, then exaggerate’ strategy has become dominant, even if only tacitly,” according to a recent academic review. “To get published in leading journals, to be awarded grants and to be hired as a postdoc or faculty member, a system-wide bias for novelty, exaggeration and storytelling has emerged.”
