Austrian economists should take an interest in the so-called “replication crisis” in science, which is affecting primarily the field of psychology, but is likely under-recognized in other social sciences—and in economics in particular.
Over the last several, an increasing number of reports have highlighted the fact that scientists are frequently unable to replicate the results of prior experiments in psychology, even when those experimental or correlational studies were conducted and analyzed according to commonly accepted methods. The magnitude of the problem was made concrete in a 2015 Science paper by lead author Brian Nosek. Replication rates, in terms of statistical significance or effect size, were only between 25-50% (depending on the particular field of psychological study).
The doubts raised by Nosek and his collaborators are metastasizing to other human sciences, including medicine. Reports of poor research reproducibility are now appearing routinely in scientific journals. Interestingly , in the context of such a replication study pertaining to the social sciences and published this past August, it was discovered that scientists are able to predict a priori with a high degree of accuracy which particular result will be replicated and which will not!
A perhaps even more startling study was just published last month. Nosek submitted a very rich data set, taken from the 2012-2013 soccer season, to 29 different teams of statisticians for analysis. The question to be answered by the statisticians was whether a darker skin tone increased the likelihood that a player would receive a red card. The variability in the methods chosen and in the results obtained was astounding, especially since, at one point in the process, the teams of analysts were encouraged to compare notes and give each other feedback.
We interviewed Nosek on our podcast about his work. It was a fascinating conversation. As an empiricist, he remains hopeful that, perhaps through greater transparency, sharing of data, and collaboration among scientists, the replication crisis will resolve itself. And he expects that the resolution will occur while preserving the standard theoretical assumptions of empiricism.
I’m not so “optimistic,” and I suspect that the crisis will be protracted. But I am hopeful that, over time, social scientists will come to recognize and accept that certain types of knowledge can only be obtained by human judgment, rather than by measurement—provided that the judgment and reasoning proceed logically from a basis of reasonable assumptions. This is a good opportunity for Austrians to showcase their methodological alternative to the scientific world
