Sometimes we may want to formally talk about objects that actually exist in the world, using mathematical language. One way to try to do this is by recording some sequence of observations about the world, and then applying Solomonoff induction. The hope would then be: if you apply Solomonoff induction to the sequence of things you’ve seen so far, it will correctly predict whatever you next see. In this post I’ll describe a problem with applying this approach to anything particularly important.
[This post contributes nothing new.]
Consider the sequence of bits observed by a camera situated within the physical universe (which we can imagine as a CA for concreteness). If we draw a program uniformly at random (i.e., fixing a universal prefix free encoding) and condition on agreement with this prefix, what does the posterior (over programs) look like?