Sometimes greatness is hard to spot. Before going on to lead the Chicago Bulls to six NBA championships, Michael Jordan was famously cut from his high school basketball team. Scientists often face rejection of their own—in their case, the gatekeepers aren’t high school coaches, but journal editors and peers they select to review submitted papers. A study published today indicates that this system does a reasonable job of predicting the eventual interest in most papers, but it may shoot an air ball when it comes to identifying really game-changing research.
Studying peer review is difficult due to the confidential nature of the process, but sociologist Kyle Siler of the University of Toronto in Canada and colleagues were able to examine the peer-review history of 1008 articles that were submitted to three elite medical journals: Annals of Internal Medicine, The BMJ, and The Lancet. In total, just 62 of the manuscripts were accepted (6.2%), confirming just how difficult it is to be published in a top-tier journal. Editors “desk rejected” 722 of the manuscripts, meaning they never made it to the journal’s peer-review stage. However, 757 of the initially rejected articles eventually went on to be published elsewhere. This allowed Siler and his team to analyze if, like Jordan, the vetoed papers would go on to achieve greatness.
The researchers found that, by and large, the gatekeeping system was predictive of a paper’s eventual number of citations. Papers that were accepted outright by one of the three elite journals tended to garner more citations than papers that were rejected and then published elsewhere, they report online today in the Proceedings of the National Academy of Sciences. Additionally, papers that were desk rejected went on to receive fewer citations than papers that were approved by an editor, but then rejected during the subsequent peer-review process. “It’s a sign that these editors making snap decisions really quickly still have a nose for what quality is and isn’t,” Siler says.