The Rationalist Conspiracy

In a famous paper, Nick Bostrom outlines what he calls the Simulation Argument:

“A technologically mature “posthuman” civilization would have enormous computing power. Based on this empirical fact, the simulation argument shows that at least one of the following propositions is true: (1) The fraction of human-level civilizations that reach a posthuman stage is very close to zero; (2) The fraction of posthuman civilizations that are interested in running ancestor-simulations is very close to zero; (3) The fraction of all people with our kind of experiences that are living in a simulation is very close to one.”

#1 and #2 seem unlikely. #1, because we haven’t found any strong reason to think existential risks are nearly impossible to avoid (see Scott’s Great Filter post). #2, because independent convergence across many possible worlds generally requires world trajectories to be very predictable, and we don’t observe that on Earth. (For example, small timeline changes might have created sentient dolphins, and dolphins have very different drives and moral systems from humans.) Therefore, most attention has focused on option #3.

As Bostrom says:

“In some ways, the posthumans running a simulation are like gods in relation to the people inhabiting the simulation: the posthumans created the world we see; they are of superior intelligence; they are “omnipotent” in the sense that they can interfere in the workings of our world even in ways that violate its physical laws; and they are “omniscient” in the sense that they can monitor everything that happens.”

However, like the gods of mythology, these gods run into what is called the Epicurean paradox, after the Greek philosopher who invented it. The paradox runs:

“Is God willing to prevent evil, but not able? Then he is not omnipotent.
Is he able, but not willing? Then he is malevolent.
Is he both able and willing? Then whence cometh evil?
Is he neither able nor willing? Then why call him God?”

For convenience, let’s number each of these possibilities 1 through 4. For a race of posthuman simulators, we can essentially rule out #1 and #4; they can probably do whatever they please.

The cynically minded might jump to #2 – the “gods” (simulators) are real but malevolent, willingly allowing disease, famine, the Holocaust, and all of humanity’s ills. But this presents another problem. A truly malevolent god could create much more suffering than we actually see. For example, it is widely agreed that electricity makes human life more pleasant. A malevolent simulator could then cause all power plants to be economically impractical, which worsens human life without affecting most other goals it might also be pursuing.

One might then postulate a simulator as merely indifferent to suffering, with goals that are entirely orthogonal. But this too would likely wipe out all the good things in human life, not just a few of them. A full explanation of why would be very lengthy, but Bostrom has described the various issues in his book Superintelligence, coining the terms “perverse instantiation” and “infrastructure profusion” for two of the most serious. In essence, almost all apparently “neutral” goals would, if carried to completion, create a universe morally indistinguishable from one where humanity is extinct. Humanity is not extinct, so we can likely also rule out #2.

For #3, one can try to invent various explanations for why the evil we see is an illusion. Perhaps only you are ‘fully’ simulated, and starving children are merely ‘zombies’ without moral value. Perhaps the simulated world was created recently, and so the world wars and other disasters never really happened. Perhaps the simulator “switches off” consciousness when people are suffering too much. However, all of these suffer from issues of Occam’s Razor: they postulate additional complexity which is inherently unobservable. The problems here are those which cause us to disbelieve the theory of Last Thursdayism, which postulates the universe was created last Thursday, but with memories and other signs of older age already in place.

In fact, observing a large number of ancestor simulations places extremely strong constraints on the goals of the simulator – essentially an even stronger version of the Epicurean paradox, or for that matter of the FAI problem. Solving the FAI problem requires formally specifying a utility function which doesn’t wipe out humanity, a tiny target in a vast space. Creating a simulator requires specifying a utility function which literally never intervenes across a vast variety of simulated situations, a much smaller target still. (One can of course speculate that the simulators intervene and wipe our memories afterwards, or some such, but this shares the problems of Last Thursdayism.)

(Another possibility, more fun to think about, has occurred to me. It seems likely that the space of human values is not large enough to fully satisfy our novelty desires over the next eleven trillion years. Since evolution is the ultimate source of our values, I have wondered if future civilizations might simulate new species evolving to sentience, so as to acquire a richer set of values than they started with. However, on reflection, it is extremely unlikely that an ancestor simulation is the best way to achieve this. Some form of directed evolution, or possibly an even more complex optimization process not yet known to us, would almost certainly be more efficient.)

By itself, this seems to be an argument for Bostrom’s scenario #2. A perfect ancestor simulation, with no intervention by the simulators, requires hitting an extraordinarily small target in utility function space. Hence, it’s not surprising that many different dissimilar worlds failed to hit it, any more than it’s surprising if a thousand gangsters shooting at random fail to hit an acorn seven hundred meters off.

However, believing in Bostrom’s scenario #2 presents a different challenge, outlined by Jaan Tallinn in his talk at the Singularity Summit. If we are not being simulated, then we are some of the very first beings to ever exist, part of the tiny fraction to live before the creation of self-modifying intelligence. The number of beings which might ever exist is truly vast, possibly on the order of 10^70. This creates another conundrum. Why should we be living now? What makes us so privileged?

I have a speculation which addresses this question. Suppose there is a shortage of bread, in your city of two million people. The city government creates a giant queue to buy bread, and assigns each resident a place in it at random. If you are placed at the very head of the line, you would think this demanded explanation; it is very unusual. Perhaps your brother is the mayor, and rigged the lottery. Perhaps you are religious, and prayed very fervently. Something must be going on; conditioning on all your other life experiences, you having this one experience is still very unlikely.

On the other hand, suppose you are graduating from college. Proudly, you walk across the stage, shake the dean’s hand, and receive your diploma. By itself, this is just as unusual as the first scenario. A college education has about two million minutes, of which only one is the one when you receive your degree. Yet, even though you may be very emotional, you don’t see the fact of living out this minute as something that demands explanation. You don’t postulate divine intervention, or an unknown friend in the administration. (Unless, of course, you are a very poor student!)

Even though it is extremely improbable, that one minute where you get your diploma is made logically necessary by the other four years of your education. Conditional on all your other experiences happening, it is extremely likely that you experience this one too; every four year project must have a first minute and a last minute. Therefore, you are not surprised.

Our standard, ancestral view of life sees people as discrete entities. A person is born, lives for a while, and then dies, with inter-human bandwidth of about 300 baud being negligible compared to intra-human bandwidth. One might envision each life as strands of spaghetti, strewn throughout a football field of time. Each strand is distinct, each has a beginning and end, and if you select one at random it is very surprising to pick the first.

However, there is no reason for posthuman civilization to be like this. When humans have a very complex computer program, it is already rare to just throw it out entirely. More likely, one creates a new version, ports it to new hardware, or adapts it for a new purpose (Windows 8’s ancestry goes back thirty-five years, and Linux’s over twenty), because code is easy to copy. In addition, when we do throw code out, almost always the motivation comes from the extremely rapid changes in computers produced by Moore’s Law; it might, for example, be easier to rewrite from scratch than to alter code to handle 100x the previous number of requests. In a world of static computers, such things would become rarer still, and even rarer if one assigned code moral value and the code did not want to ‘die’. Posthuman life would look like a single, continuous river, twisting and branching and growing through the eons.

If we suppose this, then living in the first few years of the river does not seem so surprising; every stream of consciousness must have its beginning, just as every college degree or career or sea voyage must have its first ten seconds, and the existence of the first year is necessarily implied by all of the others. Moreover, if one supposes the posthuman transition (or aging escape velocity) is likely to occur soon, this appears to solve another paradox: why we exist in the year 2014, rather than as one of the hundred billion primitive humans who lived millennia ago. If we are the first generation to be uploaded, then the stream starts with us, rather than all our ancestors who were unlucky enough to have their brains rot in the ground.