This knee-jerk evidence-based mindset is not just a pandemic thing, and it shows up wherever we treat one kind of data as the only kind that counts. In 2018, a top medical journal published a randomized controlled trial (RCT) finding that parachutes did nothing to prevent death or injury when people jumped from an aircraft. The catch was that the planes were parked on the ground, and the mean jump altitude was about half a meter. The whole study, of course, was a joke. The authors were not against experiments as such, but they were mocking the common reflex among their colleagues that treats RCTs as the only respectable form of knowledge, even about a claim you could check by looking out the window of a plane.2
I have spent much of my professional life contributing to and asking for better evidence in immigration debates, usually to the quiet exasperation of people on my own side, so I am not about to start sneering at data and acting on “vibes.” Good policymaking does need evidence, cost-benefit analysis, and careful counterfactual thinking. But it also needs humility and better judgment about what kind of evidence a given question can actually require, if any.
There are some policies that are so obviously good that we should not need a perfect study to try them, like legally allowing more housing where demand is high, keeping reliable low-carbon power online when the substitute is fossil fuel, or increasing visas for foreign top talent that everyone says they want. Reasonable people can argue about details and trade-offs. But on questions like these, the case for action does not depend on a perfect RCT, and the burden of proof should not be infinite.
There are also policies so obviously bad that we should not need a study to stop them, like asylum seeker work bans. Seriously, you do not need a randomized trial, or any hard evidence for that matter, to predict what happens when you forbid a willing adult in a legal limbo to work, pay to shelter them instead, and then point to their idleness as proof the system is broken. Most of the hardest calls in policy look more like this than some great mystery requiring a well-designed experiment.