Friendly AI From Group Tools
On Human Singularity, I said that I thought that Friendly AI might arise out of Group Tools that groups of people use to keep things just amongst their members.
These group tools, today, are things like:
- Voting Boards - people can submit things for vote, and a vote is taken
- Software that referee's procedure (For example, Roberts Rules of Order - software that automates it)
- Rubber-stamp collecting - when such-and-such people authorize something, it happens ("decision objects")
I believe that as time develops, our presently primitive software will CoDevelop with us. We have right now only very simple software, like voting boards, and rubber stamp collecting, and some experimental software that understands the concepts of stages of dialog.
But in the future, I believe that we will have communications software that will be "smarter." If someone isn't present for a meeting, and their vote is critical, then the vote will be delayed until they get back. Even better than that, the software will be scanning the tension in posts, and working to recognize when a vote seems likely to happen. If a person is about to leave for a (uninterruptable!) vacation, the software may prompt people: "So-and-so is about to leave. Maybe you want to make a decision on subject X before he goes?"
I can imagine that software with social rules will be gamed in the future. Will this mean that people will throw their hands in the air, give up, and say, "We'll just do it by hand?" I don't believe so. I believe that the software will be fixed. And it'll be "fixed" in such a way as to recognize the motions of power. That is, it'll be fixed to recognize when an individual is gaining progressively more power over the group, or whatever. Perhaps people weren't paying attention, as someone obtained control of key points of infrastructure, or something. The software would say: "Polite warning: There is a user that may upset the status quoue of the power relationships here. Is this okay?" It may politely suggest that the person take on different responsibilities, or that someone else fulfil a particular role, or something like that.
Then it seems natural to me that as trusted machines (with various bondings attached to them) took roles, that the software would be applied to those intelligences as well, just as if they were members of the society.
There would undoubtedly be accidents, and stuff like that. But people (and/or AIs) would work to fix them, I think.
I cannot be sure, and this is not a "main point" of mind. But it seems like something that would grow into "Friendly AI."
One thing about it is that it relies more on CoDevelopment than on Design. That is, as we find ourselves with increasingly powerful intelligences, we change ourselves and we change our machines in such a way that we adapt to one another in harmoneous ways.
Marshall Brain described something like this in Manna; They had a computer system that they had taught the rules of the society. I believe in his story, it eventually got to the point where the computers understood the underlying ideas, perhaps even more deeply than humans, behind the system it was upholding! But I don't remember so well.
So, this is the idea.
-- Lion Kimbro
You do not get Friendly AI except by developing Friendly AI. It is like expecting to get a Saturn V rocket in the course of developing a toaster oven.
True, but you might get a social change significent enough to count as a weak Singularity, in that we cannot make useful predictions about the dynamics of a society with markets approaching perfect efficiency, powerful collaboration tools, AI-driven search engines working on an expanded semantic web, much more intelligent and widespread software automation etc etc. Alas the uncontrolled takeoff risk increases steadily in that scenario; it is the proverbial path to destruction.
-- Starglider
Two things:
- I think the mention of the Saturn V rocket is actually rather appropiate.
After all the Saturn V was a refinement of prior designs. The prior designs themselves merely refinements of prior designs. Those designs merely refinement of still prior designs.
Making a Friendly AI from scratch is actually more like trying to make a Saturn V from a toaster, than is what I am describing.
We already have highly intelligence autonomous agents that need to be balanced against each other. They are called "human beings." And we have spent a long time figuring out ways of balancing them against one another.
Second:
- Vernor Vinge has a section on "Other Paths to the Singularity: Intelligence Amplification" -- it seems to be the same as what I'm calling the Human Singularity.
Note Vernor Vinge's fear of humans augmented by computers forming an elite.
The solution to this fear is for people to talk about "fairness" in software.
This is already happening. We see primitive group mediation (in Slashdot,) group boundries (in notions of fluid gradients in membership,) relationship awareness (FOAF and related technologies,) punishment and forgiveness (see Forgive And Forget In Software) components in software. We are already putting values into our collaboration software. I believe it is reasonable to expect this to continue, and to expect this to meld into what we grow to call Friendly AI. AI may be expected to participate under the same rules and constraints as individuals, and groups will be allowed to mediate against AI's who's actions they consider to be harmful or inappropriate. I believe these systems will CoDevelop with us.
Starglider: I've argued on Human Singularity why I think it's a strong improvement process that we're seeing amongst groups of people: The form that our interactions take (as humans) are fundamentally changing. Within a single person, the types of thinking going on are not changing. But within groups of people, technology rewiring is fundamentally changing things.
Consider Linux development: Could Free Software Linux development take place without the Internet? No, it wouldn't work with people just shipping CD's back and forth by snail mail. The interactions that the Internet made possible made the nature of our communications fundamentally different, in such a way that Linux could emerge.
But there is a stage beyond this:
Consider Wikipedia: With Internet technology, but not wiki technology, could people have made Wikipedia? No. It would not work with people just sending e-mails and ftp'ing documents to each other. It actually took the development of wiki technology for us to be able to order our communications in such a way that an encyclopedia could be made. (Nupedia was much too slow; akin to attempting making Linux by mailing CD's to one another.)
There are likely many more major changes in our arrangement of communications between ourselves, and I believe we will see substantially greater results coming out of making major changes in how our social Hive-Mind functions. Developing communications technologies = Rewiring the Social brain.
I have difficulty seeing the difference between the Social Brain rewiring itself, and an AI rewiring itself. Both have to do with fundamental changes in how they process thought.
Again: Within individual humans, there is no change. But considering the operation of the Social Brain, there is major change. We know this because we see the evidence of it.
(We should probably isolate this whole discussion onto an independent page.)
-- Lion Kimbro
Communications technology changes the topology of human communication, throughout history closer and closer to a speed-of-light mesh but recently also supporting various sorts of VLAN-equivalents (virtual communities). Collaboration and search technology add (various sorts of) intelligence to the network. Finally more and more automation interfaces allowing services to be accessed and delivered over the network are being added, with less and less human intervention. Finally archives and software tools provide the equivalent of lots of secondary storage and special-purpose co-processors. However these technologies (which in my delusion I like to refer to as 'puny earth science' :) ) can neither restructure or overclock the primary computing elements, the human beings. Adding humans sometimes works, but is subject to varying degrees of parallelisation overhead some of which is irreducable by technology. AGI and to a lesser extent neurohacking bring a new meaning to 'disruptive' (i.e. they count as mad science :) ) because they do change the speed and nature of the core computational elements. -- Starglider
