Intuitively, it seems that any simple hypothesis should receive a reasonable prior probability. It seems unreasonable to assign a simple hypothesis a ridiculously low probability. In some sense this is a matter of preference, but here I’ll briefly argue:

If H is a simple hypothesis and the prior probability of H is very small, then there is nothing you could possibly see which would convince you of H.

Which suggests that this is at a minimum a very *dangerous* thing to do (though I’m open to the possibility that there are things you should simply never believe, no matter what you see).

The argument is basically a formalization of the platitude that you can only ever get so much evidence, because hey, it’s not that unlikely that you are crazy. In fact that claim is false, as Jacob Steinhardt was quick to point out when I made it recently. The possibility that you are crazy or in a simulation doesn’t preclude you making arbitrarily large updates. It just precludes making arbitrarily large updates in favor of simple hypotheses.

First, by “simple hypothesis” I mean a hypothesis which has a short description in whatever internal language you use to represent hypotheses. So “God exists” and “God doesn’t exist” count, so long as your brain assigns them subjective probabilities.

Second, by “prior probability” I really mean the probability you assign at some arbitrary point in time, in particular after updating on all of your introspective access. Then the result will imply that at no *future* time can you come to believe H, without forgetting the evidence you have already encountered.

Third, by “very small” I mean much smaller than 2^{-|H|}. And by “much smaller” in that sentence I mean by a large factor (which doesn’t depend on H). 2^{100} should be plenty.

Fourth by “convince of X” I mean “cause you to believe X as the result of a correct Bayesian update.”

Finally, I’m going to make the assumption that you don’t assign negligible probability to skeptical scenarios (like the simulation hypothesis, insanity, or your experiences being an elaborate hoax). This can be justified by observing that such skeptical scenarios apply to a non-negligible fraction of observers in many reasonable worlds (for example, physical universes like ours). In fact, in many reasonable worlds skeptical scenarios apply to all observers. So in order to assign skeptical scenarios negligible probabilities, you’ve got to assign a whole lot of otherwise reasonable worlds negligible probabilities.

Having made those definitions the claim is now pretty straightforward. Consider some arbitrary evidence E you might receive, which might constitute substantial evidence in favor of H. I’ll exhibit a hypothesis Q such that P(H) << P(Q) and P(E|H) = P(E|Q), where P is your prior. This implies that P(H|E) / P(Q|E) = P(H) / P(Q) << 1, so in particular P(H|E) << 1.

Q is the hypothesis: H is false, however your experiences are manufactured according to the distribution P( * | H). Clearly then P(E|Q) = P(E|H). To see that P(Q) >> P(H), consider the claim S: *a hypothesis H’ was selected uniformly at random using my internal representation of hypotheses, and then my experiences were manufactured according to the distribution P( * | H’)*.

If you assign skeptical hypotheses reasonable probability, P(S) should not be not too small. Moreover, P(Q|S) is at least 2^{-|H|}. Thus P(Q) >> P(H), as desired.

Is this a fair summary of your argument: suppose H is a simple hypothesis to which I assign a really small prior probability. Then seeing evidence of H is actually better evidence that I am in some sort of simulated reality where H only appears to be true.

If so, I am confused about how one goes about even determining the meaning of H, let alone judging whether or not it is true, in sufficiently strange possible worlds, and maybe you can clear up this confusion for me. Suppose H = “the sun will not rise tomorrow” and that I assign to this hypothesis an extremely low probability. Tomorrow I observe that the sun has not risen. This appears to be evidence that the sun has not risen, but you say to me “aha, it’s actually evidence that you are living in the Matrix and the Matrix Lords are screwing with you, the actual sun totally rose.”

Okay, but if I’ve lived in the Matrix my entire life, then I learned the word “sun” in the Matrix, and if I dereference that pointer it actually refers to the part of the Matrix that simulates the actual sun, and that part really didn’t do the operation that corresponds to rising! I have never seen the actual sun. There is nothing in my current vocabulary that can reasonably be said to refer to the actual sun. So in what sense is H false here?

That’s a fair summary.

Indeed, for hypotheses like “the sun will rise tomorrow,” it’s not clear what it means for that hypothesis to be “true,” since it depends on some model of “the sun”, “will rise”, and “tomorrow.” I think this is a common domain for philosophical debate, but I know little about that debate.

I don’t think you should view sentences in isolation as things that get truth values. My take is that you have a few better-defined alternatives: you could talk only about your own direct experiences, you could talk about mathematical facts, or you could talk about statements about the world which don’t make any ontological assumptions (where the last one is still philosophically fraught).

So you might have a belief like “I won’t see the sun fail to rise tomorrow,” which can be defined purely in terms of your own experiences (in a way that is robust to ontology shifts like the one you described—a formal definition of “see the sun rise tomorrow” could be applied directly to possible worlds where you discover you are in the matrix, to pass judgment on whether or not you saw the sun rise, after you woke up from the matrix). Or you might have a belief like “in simple computer programs, observers like me live in bits of code that can be functionally modeled like ,” which is a mathematical fact (where involves a model of the earth and sun where the sun doesn’t rise tomorrow). Or you could have a belief of the form “The universe is made of space full of fields, and some of them are related to each other like .”

In any of those cases, learning you are in the matrix would just cause your belief to be automatically false.

So I agree that if I talked about my own experiences in a world where I was recently put into a simulation against my will and without my knowledge, “I will see the sun rise tomorrow” would be false because my notions of seeing and the sun both derive from a level of reality above the simulation. This is much less clear to me if I have lived in the simulation my entire life, either because I was born into it a la the Matrix or because I am actually a simulated being who has no independent existence outside of the simulation; in that case, my notions of seeing and the sun both derive from the simulation itself, so I think I would still mark “I will see the sun rise tomorrow” as true if I see the sun rise in the simulation even if the actual sun failed to rise.

I agree that something odd is going on when we talk about the beliefs of matrix-dwellers. I think for mathematical statements, or predictions about your future observations, you can make my argument go through. I’m not prepared to admit other classes of statements as meaningful on their own (instead adopting something like model-dependent realism), and I’m even a bit shaky about “predictions about your future observations,” though that’s more a statement about our collective philosophical ignorance than my own radical skepticism.

What if someone also assigns hypothesis S (that it is being simulated to fool it to believe something false) a probability disproportionately small?

Does H have to be a simple hypothesis for this argument to hold? It looks to me like the math goes through as long as P(H) << 2^(-|H|). Is that right?

That’s right—it’s the combination of improbability and simplicity that makes a hypothesis unbelievable.