<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
	>

<channel>
	<title>Ordinary Ideas &#187; meta</title>
	<atom:link href="https://ordinaryideas.wordpress.com/category/meta/feed/" rel="self" type="application/rss+xml" />
	<link>https://ordinaryideas.wordpress.com</link>
	<description>As advertised</description>
	<lastBuildDate>Wed, 31 Dec 2014 03:48:40 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
<cloud domain='ordinaryideas.wordpress.com' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
<image>
		<url>https://s2.wp.com/i/buttonw-com.png</url>
		<title> &#187; meta</title>
		<link>https://ordinaryideas.wordpress.com</link>
	</image>
	<atom:link rel="search" type="application/opensearchdescription+xml" href="https://ordinaryideas.wordpress.com/osd.xml" title="Ordinary Ideas" />
	<atom:link rel='hub' href='https://ordinaryideas.wordpress.com/?pushpress=hub'/>
	<item>
		<title>Motivation</title>
		<link>https://ordinaryideas.wordpress.com/2012/04/27/motivation/</link>
		<comments>https://ordinaryideas.wordpress.com/2012/04/27/motivation/#comments</comments>
		<pubDate>Fri, 27 Apr 2012 23:21:35 +0000</pubDate>
		<dc:creator><![CDATA[paulfchristiano]]></dc:creator>
				<category><![CDATA[meta]]></category>

		<guid isPermaLink="false">http://ordinaryideas.wordpress.com/?p=196</guid>
		<description><![CDATA[This blog contains a lot of material that is very weird. A natural question is: why write about such weird things? Many of the situations I&#8217;ve considered or problems I&#8217;ve written about involve improbable hypotheticals. We probably won&#8217;t ever have a powerful utility maximizer which we must provide a precisely defined utility function, for example. [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=ordinaryideas.wordpress.com&#038;blog=30479764&#038;post=196&#038;subd=ordinaryideas&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
				<content:encoded><![CDATA[<p>This blog contains a lot of material that is very weird. A natural question is: why write about such weird things? Many of the situations I&#8217;ve considered or problems I&#8217;ve written about involve improbable hypotheticals. We <em>probably</em> won&#8217;t ever have a powerful utility maximizer which we must provide a precisely defined utility function, for example. Why not focus on designing systems that are likely to actually get built?</p>
<p>The reason should not be so unfamiliar to a mainstream academic: though these problems may be too simple to be directly applicable, thinking about them sheds light on fundamental philosophical and mathematical issues. The heuristic is something like: where there are <em>simple</em>, <em>fundamental</em> problems on which it is possible to make progress, you should do it, even if you can&#8217;t see exactly why or how that progress will be useful.<span id="more-196"></span></p>
<p>I don&#8217;t know what the future will look like. But I&#8217;d guess it will be influenced profoundly by the development of machine intelligences of some sort. What kind of understanding is necessary, to ensure that this influence is positive? I don&#8217;t know that either, but it seems quite likely that we will have to face much hairier, real-world analogs of questions like &#8220;if we had a powerful utility maximizer, what utility function might we want to give it?&#8221; or &#8220;if we had an oracle that could answer formally specified questions, what would we do with it?&#8221; Maybe working on these simpler questions is a waste of time&#8211;maybe we should wait until we understand better what the real questions will be&#8211;but there aren&#8217;t that many simple questions in the world. As long as we only waste time on fundamental problems, we at least won&#8217;t waste too much time. And it seems to me that there may be relatively deep theory underlying these questions&#8211;if that&#8217;s the case, we may not want to count on our ability to figure it all out when the time comes.</p>
<p>This isn&#8217;t so different from the state of affairs in any other theoretical field. Why do we care about having a polynomial time algorithm for secure multi-party computation? It <em>probably</em> isn&#8217;t going to ever get used (and I could use more extreme examples, which definitely won&#8217;t get used). But because the question is so simple, and seems fundamentally related to the <em>sort of thing</em> we are likely to want to do in the future, we expect (I think rightly) that a deeper understanding is likely to be independently useful, and to suggest fruitful directions for further study.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ordinaryideas.wordpress.com/196/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ordinaryideas.wordpress.com/196/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=ordinaryideas.wordpress.com&#038;blog=30479764&#038;post=196&#038;subd=ordinaryideas&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>https://ordinaryideas.wordpress.com/2012/04/27/motivation/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="https://0.gravatar.com/avatar/0a58766009bacac271106a2f92289d90?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">paulfchristiano</media:title>
		</media:content>
	</item>
	</channel>
</rss>
