Comments on: Avoiding Simulation Warfare with Bounded Complexity Measures

By: Specifying a human precisely (reprise) | Ordinary Ideas

Specifying a human precisely (reprise) | Ordinary Ideas — Sun, 24 Aug 2014 21:43:40 +0000

[…] significant concern with this procedure is the one I discussed before, essentially that Solomonoff induction might end up believing the simulation argument and so this […]

By: Solomonoff Induction and Simulations « Ordinary Ideas

Solomonoff Induction and Simulations « Ordinary Ideas — Thu, 24 May 2012 16:56:42 +0000

[…] obvious way to avoid this sort of thing is to avoid the universal prior. I mentioned before the possibility of using a prior which penalizes algorithms which use a lot of time or a lot of […]

By: paulfchristiano

paulfchristiano — Sun, 25 Dec 2011 06:21:50 +0000

If you are using the universal prior, the AI impersonator (say) doesn’t ever have to run and gets access to unlimited resources (the theoretical output is then just a mathematical abstraction about which our AI reasons).

If you are using something like the speed prior and dealing with the concerns of the post, then the issue is just that pointing to a human in the universe is probably harder than pointing to something just a little later in history when the universe has been (spatially) tiled with computronium or what have you. An AI living in our future will have a more direct way of learning about the experimental setup than simulating the universe, so no computational limitation on such an AI will help avoid this failure mode.

By: endoself

endoself — Sun, 25 Dec 2011 05:53:36 +0000

Do you think it is likely that an AGI would be able to find a human given a program describing the universe with a realistic amount of resources or are you considering this possibility even though it is unlikely? I think that it is something we need to be prepared for, but you seem to me to think that it is likely, which surprises me.