Wiki Interview With Eliezer/Cognitive Science

Cognitive Science

(Section needs more questions.)

Why is studying, understanding, and advancing Cognitive Science, such as cognitive neuroscience and Evolutionary Psychology, more important than doing the equivalent in subfields of AI? Why is Cognitive Science research so important to the success of AI?

You have to invest in both. It's true that I think there are certain subfields in AI that are either mined out, or more properly never had any ore to begin with, while in the Cognitive Sciences there is steady progress. One might even expect the Cognitive Sciences to swoop in and rescue AI, at such time as the Cognitive Sciences would have tackled the problem on their own if the computer scientists had never gotten into it. One might even look at the reference list of "Levels of Organization" (DGI) and say that this is just what has happened, but at some point you do need to apply the knowledge from the Cognitive Sciences to new attempts to build AI. Otherwise there's no exclamation point at the end of the sentence, as it were, but I do think that in a fundamental sense, contemporary Cognitive Science works, and what is contemporarily called "AI" does not, with one or two exceptions such as attempts to build artificial sensory modalities, and that attempts at *real* AI should therefore be built on the base of knowledge that is the contemporary Cognitive Sciences. Of course, this requires being able to tell the difference between real grounding and magical analogies - an example of the latter being "This neural network is just like a human's brain! It's parallel!"

What open questions in Cognitive Science have strong relevance to your potential success of engineering General Intelligence?

The way in which concepts control mental imagery, I think, is something that has not yet seen a lot of study because you have to know what you're looking for. The decomposition of *cognition* and not just *perception* into functional subsystems is a fundamental and important research area that has not seen a lot of progress because the hypotheses haven't been good enough to say where to look. Cognition is fundamentally perceptual, of course. I mean that people have been looking at functional decompositions of "perception" more than they have been looking at decompositions of "cognition", not that these are actually two different things. I'm talking about attitudes. Specific analyses of the algorithms performed by biological neurons are also extremely important, since they tell us what it is we need to port over to hardware. For example, the architecture of the cerebellum looks *very* suggestive if you're thinking about how to do learning of error correction in realtime process control. Evolutionary Psychology - in the broad sense of "evolution of all cognition", which is what the field is really about, and not just the sexual-morality stuff that's all you see in the newspapers. Evolutionary Psychology can tell us about how human minds evolved, provide constraints on which systems evolved and in what order, tell us about how evolution incrementally improved the human design, and so on, and so on. If I had to identify two top priorities, it would be the neurocomputational investigation of real biological neural networks, and the neuroanatomical decomposition of functional elements of General Intelligence (for which you need a hypothesis about functional elements of General Intelligence, please note.) Evolutionary Psychology needs to investigate the evolution of moral reasoning in linguistic organisms - that is, humans. That's something for which you need hypotheses about the cognitive processes of moral reasoning, so that is interrelated to Friendly AI in the same way that AGI (Artificial General Intelligence) relates to study of the decomposition of General Intelligence in humans.

You have written, "The Prime Directive of Artificial Intelligence: Never lie to an AI, and never attempt to control it except with truthful logic." Is this still valid circa 2002? Why or why not? Why will we able to "control" AI with "truthful logic"?

Again, this is obviously obsoleted by Creating Friendly AI, But damn right you'd better never lie to an AI. If you lie, the AI eventually spots the lie, then your little house of cards collapses. This is not a toaster oven we're talking about creating here.