Science has been peculiarly resistant to self-examination. During the ‘science wars’ of the 1990s, for instance, scientists disdained sociological studies of their culture. Yet there is now a growing trend for scientists to use the quantitative methods of data analysis and theoretical modelling to try to work out how, and how well, science works — often with depressing conclusions. Why are these kinds of studies being produced, and what is their value?
Take a study published on 10 November1 by psychologists Andrew Higginson of the University of Exeter and Marcus Munafò of the University of Bristol, UK. It considers how scientists can maximize their ‘fitness’, or career success, in a simplified ecosystem that allows them to invest varying amounts of time and effort into exploratory studies. The study finds that in an ecosystem that rewards a constant stream of high-profile claims, researchers will rationally opt for corner-cutting strategies, such as small sample sizes. These save on the effort required for each study, but they raise the danger that new findings will not prove robust or repeatable.