Experimentation and Policy
I spent the past three days at the New Zealand Association of Economists annual conference where I got to hobnob with my fellow practitioners of the Dismal Science. I attended a number of interesting sessions, though since I’m an economist I have an eccentric definition of “interesting”.
One session I think you guys would be interested in though is the keynote speech by Tim Harford, based on his new book Adapt. The theme he raised in his speech was concordant with some of the thinking Erik and I have been doing lately about knowledge and policy.
Harford’s key message was that the social and policy problems that we face are unfathomably complicated – he uses The Toaster Project as an example of how even the simplest of consumer goods is the product of a vast interconnected chain of processes that defy easy description. Given that making a frakking toaster is that complicated, is it so surprising that the problems of health, education and the environment vex us so?
In comparing the effectiveness of markets vs. government he notes that markets successfully solve complex information problems that defy any panel of experts. But rather than simply stop here, and say “well, I guess markets are just better, end of story” he examines what it is markets are doing to solve these problems, and whether governments can learn to do it too (as an aside this is one of the major differences between having a conversation like this among economists and having it among politicians. Politicians tend to treat markets as magic, differing primarily in whether they view them as white magic or black magic, while economics is about trying to figure out what makes them tick).
The conclusions he reaches are very different from the Randian (or even Schumpeterian) view of the Heroic Entrepreneur who creates consumer (and producer) surplus through having a grand vision, and the will to carry it out. Instead, he goes for more of a Hayekian view, with a healthy dose of Darwin thrown in. It seems that to a large extent the private sector is just trying things more or less at random – the differences between businesses and government is what happens next. Effective organisations learn from their failures – they try many things, work out what’s working and what isn’t, and shut down the failures. This combination of variation, selection and persistence is exactly what drives evolution.
Harford recounted at one point a discussion he had with a Google executive in which the executive said Google expects 80% of their new products to fail. The expected failure rates for venture capital can be higher than that. By contrast a politician who makes even a single mistake can find themselves with a badly damaged career. The perverse political incentives politicians face (never make a mistake ever), lead to negative consequences for government’s performance. For one thing the political process selects for politicians who value certitude over humility. But it also affects how policy is implemented. Harford argues that to avoid being caught in a mistake politicians approach social problems one of two ways: A) do nothing (but that can be pretty risky too) or B) do something but ensure there’s no way to tell if it worked.
Which brings me back to education policy. We’d all like to know how to give a good education to children in adverse circumstances, but that’s not something we know how to do. So the only way to find out is to experiment. Let people set up new schools and new ways of doing things. Closely monitor their performance and learn. This will necessarily result in variable outcomes (some experiments will work and some won’t), but in the long run everyone will be better off, and isn’t that what matters?