The Rise and Fall of Science: “despite massively increasing the number of people working in academia, we’ve seen an absolute decline in the rate of major theoretical progress in most academic fields”

Ideas and data:

To begin with, it is necessary to make clear how we go about measuring scientific progress. Broadly speaking, there are two ways of quantitatively measuring this construct.

First, there is the subjective approach. This approach measures scientific progress during a time period by counting up the number of important events which occurred during that period or the number of important people who lived and made achievements during the period. What events and which people are important is determined by relevant experts.

This expert opinion can be measured in a variety of ways. For instance, you can ask experts to rate a list of potentially significant figures or events, you can analyze the frequency with which individuals or works are cited within expert material, you can analyze the frequency with which individuals are included in encyclopedias, or the amount of space they are given in such works, etc.

Obviously, this approach will not work if experts do not agree on which events/people were important, but, as it turns, out, there is a high degree of consensus between experts in a wide variety of fields.

This can be seen by looking at how different experts (both individual experts and groups of experts) rank the relative importance of individuals across various methods, including having people give direct ratings, looking at how frequently individuals are cited, or even analyzing the space each person is given in topic-specific encyclopedias. Such methods produce reliability coefficients of .86 for art and .94 for both philosophy and science. Even when comparing sources from different nations, including comparisons between western and non-western sources, there is a remarkable degree of agreement (Eysenck, 1995Murray, 2003).

So, there exists a set of people and events that experts consider to be important and there is a rank order of these people and events that experts largely agree on, and this rank ordering does not appear to be culturally biased. We can use these lists to measure scientific progress by comparing how many important people and events occur during different times or in different places.