An Economist Nitpicks Sci-Fi: Foundation and Chaos

Welcome back to An Economist Nitpicks Sci-Fi, a series where I evaluate the plot and setting of science fiction works. The main point of difference for this series is that I am examining the works not through a literary or physical science perspective (both of which are fairly common on the internet), but rather through the perspective of an economist.

In this post, I will be looking at the central trilogy of Foundation novels by Isaac Asimov: These being Foundation, Foundation and Empire and Second Foundation. Since I will be discussing psychohistory, which is an integral part of the Foundation series, I will be discussing full spoilers for these three books. Because it will be impossible to make sense of this post without looking at spoilers, these spoilers will be unmarked. If spoilers for Foundation are a dealbreaker for you, then I suggest abandoning this post now.

Foundation and Seldon

Foundation is a story about the dying centuries of a galactic empire 10,000 years in the future. More than that, it is the story of an initially small foundation of scientists tasked with returning the galaxy to civilisation as quickly as possible as the empire collapses. The eponymous Foundation works at the behest of Hari Seldon – a social scientist with capabilities as far beyond the abilities of modern economists, sociologists and psychologists as Faster than Light travel is beyond the capability of modern physicists and engineers. For Seldon has invented a new social science, psychohistory, which gives him the ability to forecast the inevitable collapse of the Empire, even though in his lifetime it still looks unassailable. And while his science also tells him that it is too late to stop the Empire’s collapse, he can see a way to reduce the period of barbarism and anarchy that will grip the galaxy for 30,000 years down to only 1,000. This is the task he creates the Foundation for, a task he will not live to see begin, much less end: to grow into a power that can bring science and order back to the galaxy as quickly as possible.

Foundation, the story and the organisation, is inherently tied up with Seldon and psychohistory. And this is an interesting twist to science fiction – normally science fiction stores involve advanced physical science, but Foundation is fairly unusual in having the key focus of the story be an advanced social science. I am not the only economist to find this interesting – Paul Krugman has cited Foundation as an inspiration for him becoming an economist.

Seldon’s ability to predict what obstacles the Foundation will face, and to make advance plans accordingly, is what drives the Foundation to start realising its purpose as the final guardian of galactic civilisation. This means that the plausibility of Foundation rests on the plausibility of psychohistory. So how plausible is psychohistory as science fiction?

First off, I will surprise no one by pointing out nothing like psychohistory exists in the world today. Many social sciences (including economics) make forecasts and predictions about future outcomes, but only in very narrow domains, over fairly short time periods and with limited success. But this is hardly a deal-killer for science fiction, after all faster than light travel, elemental transmutation, and advanced nuclear technologies are also on display in Foundation. Science fiction is about extrapolating science in plausible ways, so the question we need to ask is could a future society invent psychohistory? To answer that, we need to look at what Foundation says about psychohistory and how it works.

Psychohistory: The Foundation of Foundation

In Foundation, Hari Seldon, the pre-eminent authority on psychohistory, describes it as the application of statistical methods to history and psychology. The logic is that with sufficient historical data, and sufficient understanding of human psychology, it should be possible to build a model of human behaviour that takes current and historical geopolitical conditions as an input and produces forecasts of future geopolitical conditions as an output. These predictions would not be certainties, but they would show the likely and plausible arc of future history.

This is not a novel or radical concept – basically all forecasting works this way. Sometimes it works well, and other times poorly. A lot of a model’s performance depends on how much data the forecaster has and how well the forecaster understands the phenomenon in question. From a plausibility standpoint, I am happy to concede that a hypothetical future society could overcome the data and conceptual barriers to producing a good-quality historical forecasting model. That’s not a bigger claim than that physicists may one day build a faster-than-light drive.

There are, however, two other objections that are harder to wave away. Asimov, to his credit, does anticipate these issues to some degree, so I will be explaining each of these issues, how Asimov tries to deal with them, and how successful I think he was.

The First Problem: Adaptation

One thing that makes social science different from (and more complicated than) physical sciences is that when you study humans, they study you back. Humans are fairly unusual among the class of known phenomena in that they can read the papers you publish and change their actions or beliefs accordingly. This can result in your model trying to predict itself embedding a feedback effect – your model’s outputs get reduced to random noise.

This is not a strictly theoretical problem, as a couple of historical examples from economics and finance will demonstrate:

The Monday Effect: Once academic finance really got going, one of the notable results researchers discovered was The Monday Effect – share prices tended to be slightly higher on Mondays than on other days, accounting for as many confounds as the researchers could identify. This was important because it was a rare piece of evidence against the Efficient Market Hypothesis – a pattern like this could not be explained if the market was making proper use of the information available to it. However as research on the effect developed, later researchers found they could not replicate the earlier results. This was not an epistemic problem – the phenomenon was visible in older data, but not in newer data that had been subsequently released. What had happened is that financial markets had noticed all the academic chatter about the Monday effect, and traders had changed their trading behaviour to take advantage of it. Like with any arbitrage opportunity, exploiting the Monday Effect caused it to disappear. In other words, identifying and studying the Monday Effect caused it to disappear.

The Phillips Curve: Bill Phillips was an early 20th Century economist who identified a relationship between inflation rates and unemployment – specifically, the more of one a country had, the less of the other it tended to have. This attracted a lot of attention at the time, because this jibed with Keynes’s then-recent ideas about using stimulus to combat recessions. The relationship Phillips identified formed the basis of macroeconomic policy for much of the world for the 1940s through to the 1980s. Unemployment is a much larger social problem than inflation, so the reasoning went, and therefore governments should use stimulus to drive up inflation and thereby keep unemployment low. This worked great until the 1970s when things came to a head. The stimulus was producing inflation like it always did, but the resulting employment effects were getting weaker and weaker, leading to stagflation (a combination of high unemployment and high inflation in flagrant violation of what the Phillips Curve predicts). The solution to this paradox was discovered by Milton Friedman in the late 1960s. It turns out that the relationship between inflation and unemployment Phillips observed only applies when the inflation is a surprise to the economy. Back before government engineered inflation rates, every change in inflation was a surprise, so the relationship looked robust. By implementing stimulus, governments had changed the population’s expectations of future inflation, and thereby permanently shifted the Phillips Curve in a way that utterly frustrated the stimulus programme. The modern Expectations-Augmented Phillips Curve includes an additional term to account for people’s expectations of inflation, as a separate variable from inflation itself.

To Asimov’s credit, adaptation is something he anticipates and accounts for well in Foundation. Seldon keeps his math hidden from the general population, keeping it among his team rather than publishing it academically. Equally, when the Foundation is established, he does not share his methods with them, instead leaving them ignorant of psychohistory . He passes his knowledge on to a second foundation (imaginatively named The Second Foundation), which stays hidden from the first Foundation, refining psychohistory and Seldon’s Plan in secret. Seldon’s reasoning in maintaining this level of secrecy is precisely that wider knowledge of psychohistory and its methods would invalidate Seldon’s predictions. So I am giving Foundation a passing mark for its response to Adaptation.

The Second Problem: Chaos

The future is difficult to predict [citation needed]. Any attempt to predict what is going to happen requires taking what we do know about how causes lead to effects and using that to find the most likely effects of the existing known causes. The precise methods of doing this vary widely, and understanding even a few of them can take years of education, but broadly speaking there are two high-level structures that are used to predict something an analyst cannot model precisely. The process underlying a phenomenon’s behaviour can be described as either stochastic or chaotic.

The nature of stochastic and chaotic processes are a topic that takes years to understand fully (as it is, I know the basics of chaos theory, but no more than that), but here is a quick example that will hopefully shed some light on the difference.

Consider a spinning top on a table. How would you model the top’s movement across the table so you could forecast its position in a minute’s time? The answer will depend on a combination of large and small factors:

• There are factors that pertain to any possible spin of the top, such as whether the table is perfectly level, or has an incline at all. Call these Systematic Effects.
• Then there are factors that relate to any one spin, such as how hard it is spun, and whether the spinner imparts any bias in how it moves based on how they spin it. These are Unsystematic Effects.

This is typical of a stochastic process – a series of small effects will move the top over time but each little movement will not be dramatic on its own, so the range of probable locations of the top is fairly predictable. This gets more important if instead of predicting a single spin, you predict the average of a large number of spins of the top. The unsystematic effects on the top for each spin will tend to cancel out, meaning that only the systematic effects will matter. This makes modelling the top much easier, and the range of uncertainty around the average top’s position will grow steadily smaller as more and more spins are averaged together. So along as you understand the systematic forces that effect the spins of the top, you can predict the pattern of top spins quite reliably.

This property of stochastic processes, The Law of Large Numbers, can make them quite predictable if your sample sizes are large enough. It’s the reason opinion polls can predict elections quite well even though they only ask about 1000 people who they plan to vote for.

However, since the 1950s, further advances in mathematics have shown that not all processes follow this logic.

Let’s go back to our spinning top, but instead of a table, let’s take the top and spin it on top of an upturned bowl. The bottom of the bowl is flat, so while the top is near its origin point, it will act the same way it did in the previous example, but as soon as it hits the bowl’s sloped sides it will slide right down to the rim of the bowl and never return. This makes predicting the top’s location harder, not just because the top moves away from its origin faster but because the endpoint for the top can be radically different based on small changes in how it moves shortly after being placed on the bowl. Predicting the top’s location even vaguely depends on being able to successfully predict the first few steps the top takes away from its origin. And note that this is a fairly simple example, a surface with many slopes and plateaus at different angles would be even harder to predict. This property, metastability, is an important component of chaotic processes.

Unlike in a stochastic process, the Law of Large Numbers doesn’t help us much with a chaotic process. Let’s suppose our surface has a slight unobserved bias that causes the top to move East more often in its first second than in any other direction. On the flat table, this isn’t a big deal; sure, our estimate will be off, but it will only be slightly off. But this same bias in the bowl would have a massive effect: the increased odds of the top falling toward the eastern rim of the bowl (and reduced odds of falling to the western rim) will move the average top endpoint right over to the eastern rim. In a stochastic model small data errors lead to small prediction errors, but in a chaotic system even small errors in your initial data can ruin your model. This is why weather forecasting is so hard; you miss the air displaced by the flight of a single butterfly and your model can miss a hurricane. It also means that chaotic systems tend to have a predictive horizon – predicting a chaotic process is workable up to a certain point in the future, but becomes highly unpredictable beyond that.

So why does all of this this have to do with psychohistory? According to Hari Seldon, psychohistory is a statistical science: it cannot predict events with certainty, but it performs well at predicting the scope of human history. The logic is that human idiosyncracies are naturally smoothed out in large populations, leading to a largely predictable pattern. In other words, psychohistory operates on the premise that the progress of history is a stochastic process – some details will escape the psychohistorian’s attention, so as time passes it gets harder and harder to predict the future, but this is a gradual loss of predictive power rather than a dramatic decline.

But is history stochastic or chaotic? Consider this history of the 20th Century – the entire geopolitical structure of the world radically changed. Most of the major empires of the world either fell or were radically diminished, leading to a bipolar conflict between the USA and the USSR (one being a minor power in the 19th Century, while the other didn’t exist at all). And then in the final decade the USSR itself collapsed, leading to a monopolar environment with the USA unchallenged for dominance. What would a psychohistorian in 1899 need to know to predict geopolitics in 1999?

• The first thing you’d need to predict is World War One. And that was precipitated by the assassination of Franz Ferdinand by Serbian nationalists, so your model will need to account for every tiny band of nationalists and could-be assassins who could disrupt the balance of power in Europe.
• On the other hand, Europe was a powder keg in the 1910s. Maybe the precipitating event doesn’t matter much. One could argue that WWI was going to happen anyway, and all the Serbian nationalists did was change the timing a little. That’s a fair point, but what effect would changing WWI’s timing have? For example, what is the probability that Corporal Adolf Hitler would have been killed if the times and places of battles in WWI were slightly different? I have to assume it’s high, since the death rate of front line soldiers in WWI was very high. And what would WWII look like if Hitler had died? Would it have happened at all?
• For that matter, if WWI had happened a little earlier or later, what effect would that have had on the Russian Revolution? Would different timing have favoured the Mensheviks over the Bolsheviks? Might the revolution have been thwarted entirely? Try to imagine the back half of the 20th Century without the USSR.
• Now consider every other thing that could have affected a major world event in the 20th Century.

While Seldon never talks about predictive horizons for psychohistory (which is not surprising since the math hadn’t been invented yet), Asimov has a character in Foundation and Empire that highlights the effects that chaos would have on psychohistory – The Mule. The Mule is a mutant, infertile (hence the name), but with mental powers that allow him to control the emotions of those around him. This gives him the power to manipulate large groups of people – turn bitter foes into loyal subordinates and destroy the morale of whole planets at long range. The quirks of human biology that gave rise to the Mule were outside psychohistory’s scope, but are large enough that they don’t cancel out even at a galactic scale. One of the best scenes in Foundation and Empire has the citizenry of the Foundation gathered at Seldon’s vault to hear one of his pre-recorded predictions of where the Foundation would be at this point, and what problems it would be facing – only for it to prove that he failed to predict the Mule’s appearance, leaving Seldon’s pronouncements useless.

But Asimov doesn’t just show the effect of chaos, he also show how chaos can be dealt with – don’t try to predict the top’s spin, but rather change the shape of the surface it spins on. Seldon deliberately established the Foundation on a remote, mineral-poor planet to minimise the number of variables that could disrupt the plan. And what ends the Mule’s threat isn’t the psychohistorical forces that stopped every previous threat but the Second Foundation – a secret group of psychohistorians who covertly monitor The Foundation and intervene to keep it on track if something unexpected comes up. On top of that, the Second Foundation continues to refine both psychohistory and Seldon’s plan, giving the plan the greatest possible chance of succeeding.

From an economics perspective, Foundation was more plausible than I had expected. Don’t get me wrong, I doubt we’ll ever be able to build geopolitical models as good as Seldon’s, and Mule-like disruptions would be far more common than Asimov portrays (I also very much doubt that advanced social science knowledge will ever bestow people with psychic powers), but Asimov made a serious effort to figure out what a good social science model would and wouldn’t be able to do, and how one might deal with some of forecasting’s weaknesses.