Michael Anissimov/Distributed Working Memory
This is the page I will use to temporarily post paragraphs I'm soliciting criticism about. (Please leave comments on this page.)
Apparent ethical conflicts have had a tendency to evaporate under conditions of sufficient intelligence, resources, technology, or understanding. Fighting over resources is not necessary when you have the technology to obtain ample resources for everyone. Although some people may continue to fight just for the sake of competition, this ethical dilemma may also possess a limited lifespan; when we can engineer the very essences of who we are, we will be able to make the choice not to be competitive, ever, if we judge that as truly desirable. If the total amount of resources in we can obtain happens to be equivalent to a 10^12 units, and 10^9 individuals exist, then it will be our implicit responsibility to halt our desires at our fair share; 10^3 units. If we tune our desires to strive for 10^4 units, then we are bound to be disappointed, because our obtaining those resources would require unfairly stealing from others. Given the current human levels of desire for excitement, fun, emotions, and resources, the Solar System alone would probably provide enough to entertain quadrillions of individuals (there's a lot of matter out there, and a lot of ideas for transforming it into value-structures.) Remove the emotional atmosphere coloring most ethical discussions, and "maximizing happiness for everyone" becomes a game-theoretical mathematical task - a task where obtaining the solution leads to real-world, tangible betterment. These tasks are the sort that a true Friendly AI would be designed to begin pursuing.
"...implicit responsibility to halt our desires at our fair share..." -- at, or before. Perhaps better to phrase this as something we'd *like*, rather than a responsibility. With cognitive engineering you don't need to force yourself.
"value-structures" -- "valuable structures"? "desirable structures"?
"and[, say,] "maximising happiness for everyone" becomes a [concrete problem to find solutions for,] where obtaining the solution..." -- (1) maximising happiness is an interim approximation (2) game theory is relevant, but it seems to be more than a game-theoretical mathematical task.
- Nick Hay
