What a database of more than a thousand dismissive literature reviews can tell us

Richard Phelps:

I was once required to testify in a court case. My lawyer gave me a few pieces of advice, but he repeated one several times, which may be why I remember it. “Never say never,” he said. Or, conversely, never say always. Declarations of absolutes present opposing attorneys too wide an opening. They need to identify only a single example to contradict. In trial courts, one cannot get away with making reckless absolutist claims unchallenged.

In academic scholarship, however, it happens all the time.

Meet the dismissive literature review, in which an author at the beginning of a journal article declares the published research literature on the topic either nonexistent or so poor in quality that all of it is … dismissible. Typically, no evidence supports the claim. You’ve seen the claims yourself (e.g., “little previous research has, …” “few studies have looked at …,” “there is no research on …,” etc.). With one type of dismissive review — a firstness claim — authors boldly declare themselves to be the first in the history of the world to study a particular topic (as in, “this is the first study of …”).

In academia, declarations of a void in the research literature are rarely challenged. As long as a few unknowing, uncaring, or otherwise cooperative reviewers and editors let the statement slide, it passes unimpeded into the world of scholarship and becomes what I call a dismissive literature review. No one with a self or public interest in countering the claim is offered an opportunity to challenge.

The size of my collection of such reviews surpassed 1,000 some time ago, despite its limitation to a single, relatively small topic — U.S. education policy — and a small proportion of researchers in that field sometimes labeled “celebrity scholars.” Those are researchers blessed with public relations offices and information dissemination budgets supporting the promotion of their brand (think government-funded research centers, think tanks, the most prestigious universities).

The Malfunction of US Education Policy: Elite Misinformation, Disinformation, and Selfishness Roman & Littlefield, 188 pages.