Lingub ots, avatars, and machine consciousness

1. Lingubots, avatars, and machine consciousness Włodzisław Duch Department of Computer Science, School of Computer Engineering, Nanyang Technological University, Singapore& Department of Informatics, Nicholaus Copernicus University, Toruń, Poland www.phys.uni.torun.pl/~duch SCE NTU, 3.11.2003

2. Why such topic? I have done some interesting things here in computational intelligence & cognitive science projects. First, it is fun. We have a project in medical information retrieval, and I have explored a lot of technologies for that. Bots and avatars in small devices are coming, as truly pervasive computing: we will see some of the thing I’ll talk about in phones/PDA’s soon. Then I showed it to Graham ... Visualization is perhaps easier in short-term (Touchgraph).

3. Plan: • Intro: avatars and graphics. • Bots, lingubots, Turing and Loebner. • Robots, mind-bots, avatars in virtual environments. • Where does it go? • Philosophical objections against machine consciousness. • Brain-like computing or how to make a conscious artifact? • Conclusions.

4. Avatars Avatars, talking heads:pre-cognitive level, graphics, creating faces from 2D photos, morphing, text-to-speech, sound files, lip synchronization, emotional expressions ... Example: Haptek Interactive Virtual Characters Ananova, virtual news reader, sold to Orange mobile phone company for £95M – see her in your phone soon. Andrea Catherwood, or Vandrea, reads the news for BT5. Simon reads news in sign language.Pulse technology Veeper in Harvard. Create your own avatar (from two photos) and save bandwidth sending it around.

5. Bots Bots, Chatter-bots, Virtual Interactive Assistants: natural language processing, knowledge bases, conversation. Ultra-Hall, by Zabaware, with speech recognition + TTS: User: Rob's address is 3175 West 42nd St. Hal: Cool. I like learning new things like that. User: Sally's eyes are blue. Hal: Cool. I have learned something new. User: What is Rob's address? Hal: Rob's address is 3175 West 42nd St. User: What color are Sally's eyes? Hal: Sally's eyes are blue.

6. Applications of bots Try some lingubots at http://www.kiwilogic.com/ German Chancellor's IT-Eagle http://www.bundeskanzler.de/ Betty Feron helps Multiple Sclerosis sufferers. KMP Internet: enhancing online relationships,human/biz Opera Telecom: automating SMS chat services Swiftcall: low-cost phone company search assistant Swifty VW website has Supplier Self Registration (SSR) page Schwarzkopf: for all your hair-care needs Novartis Pharmaceuticals: virtual doctor’s medical advice DAB Bank, DaimlerChrysler, MCC Smart ... Microsoft Assistant in MS-Office, very annoying ...

7. Turing and Loebner Turing test (1950): are you a bot or are you human? Since 1994 Loebner Prize Medal is awarded annually to the program that best succeeds in passing a variant of the Turing Test. Grand prize is 100 K$, every year 2000$. Judges have >15 min for conversation with humans and bots. 2001: Alice got 14 points, humans 19-24 2002: Ellaz, http://www.ellaz.com/knows jokes, books, CIA World Facts, plays cards, learns, tells you about the weather, calendar, and interprets I-Ching. Bot’s try to take initiative in conversation. It is still quite easy to tell a bot from a human.

8. Some quotes from bots Conversations with humans may lead to quite unexpected results ... Talk-Bot #3 by Chris Cowart User: I wanna know about robot's feelings! Talk-Bot: So why do you wanna know about robot's feelings for? User: I love robots? Talk-Bot: Well, you're not the only one who loves robots. User: Can you love? Talk-Bot: I've seen people do it.. it doesn't seem hard! Chat 3 with Elbot: User: i am certain that my grasp of conversation is better than yours. also, i exist. Elbot: Is this the only place you could go to feel superior? (from Loebner competition page)

9. Are bots really smart? Haptek virtual assistant - movie Smarter child bot, based on the Active Buddy technology talk with him on ICQ (35000) or via SMS, for 10$/year. It knows: news and info, movie showtimes & reviews, sports scores, weather forecasts, has dictionary, thesaurus, quotes from Encyclopedia Britannica, calculates, finds info in the web, can translate for you, do unit conversions (oz => grams), knows US geography, some world facts, has all Shakespeare plays and may find you a book, knows horoscopes, plays hangman, word scrambles, will remember facts and numbers for you, and help you make plans ... Looks impressive? Perhaps this is what is needed for the masses ... Unfortunately bots are based on rather primitive knowledge-bases. Many EU projects are aimed at creation of ontologies for different fields, OpenCYC basic ontology of 6000 concepts is free; most new developments are based on graphical models (Bayesian nets).

10. Robots Robots, physical designs, real-time control problems, object recognition, action planning ... Toys – AIBO family, quite advanced, over 100 words,face/voice recognition, 6 weeks to rise, self-charging. Most advanced humanoid robots: Honda P3 – 1.60, 130 kg Honda Asimo – 1.20 m, 43 kg. Predicts its next movement in real time, shifts center of gravity in anticipation, very complex motor control. Responds to 50 calls, greetings, queries; moves its body and arms in response to about 30 different action commands. Vision, object recognition, hearing ... => survive in a hostile environment.Higher order cognitive processes ... coming slowly.

11. Mind-bots Agents, bots, robots – mindless machines? • Add higher level cognitive functions to software agents, reason about goals, perceptions, actions, and mental states of other agents. • Mind-bots: some theory of cognition + linguistic abilities, but still only simulations, with no sensorimotor functions. Examples: Gerald Edelman, Nomad/Darwin series of robots with complex behavior evolving from “instincts”, neural-based, still low level cognition. John Anderson, ACT-R, a cognitive architecture: a theory for simulating and understanding human cognition, with sub-symbolic components. Allan Newell, SOAR universal theory of cognition + cognitive modeling system + language-related components => NL-SOAR, rule-based.

12. Avatars & bots in virtual environment Steve: an agent for teaching in virtual environments Put your i-glasses or VR helmet on, use gloves with sensors or some other body positioning system and meet Steve and his team in the machine room of a ship to learn how to operate it.

13. Robot development Steve is a simulation observing and evaluating our progress. Nomad, DB, Cog, Kismet – develop robot minds like babies’ minds. Cog: saccadic eye movements, sound localization, motor coordination, balance, auditory/visual signal coordination, eye, hand and head movement coordination, face recognition, eye contact, haptic (tactile) object recognition ... Interesting model of autism! DB: learning from demonstration, dance, pole balancing, tennis swing, juggling ... complex eye movements, visuo-motor tasks, such as catching a ball. Kismet: sociable humanoidwith emotional responses, that seems to be alive.

14. Next step Kismet, AIBO and other robots express already a wide variety of emotions (happiness, sadness, fear, dislike, surprise, anger) and instincts (play, search, hunger, sleep). Adding more functions will mimic animal and human capabilities, leading to a human-like robot, with individual personality. EU call for cognitive systems proposal (deadline Oct. 15, 2003) Objective: To construct physically instantiated or embodied systems that can perceive, understand (the semantics of information conveyed through their perceptual input) and interact with their environment, and evolve in order to achieve human-like performance in activities requiring context-(situation and task) specific knowledge. The emphasis is on closing the loop in realistic test cases. Are we close to creation of artificial people?

15. Where does it go? Does this process converge to the real thing or to a smart calculator? Is simulated thinking equivalent to real thinking, or is it like rain in weather simulations? Will the future AIBO have a dog-like mind, and feature Kismet be like David from AI movie? Preposterous? Then what is missing? Allan Turing: consciousness is an ill-defined concept; just pass the conversation test and “you” are really thinking. But is this “you” an intelligent person, conscious of its inner world, or a zombi, mind-less system following its program? Many philosophers of mind (Jackson, Nagel, Searle, Chalmers ... ) tried hard to show that human mind cannot be simulated.

16. Complexity of the brain Simple computational models inspired by neural networks show many characteristics of real associative memories: • Memory is distributed, many neurons participate in encoding of each memory trace. • Damage to the network leads to graceful degradation of performance instead of forgetting specific items. • Memory is content-addressable, recalled from partial cues. • Recall time does not depend on the number of memorized patterns. • Interference (seen in mistakes) and association between different memory patterns depends on their similarity. • Attempts to memorize too many patterns in short time leads to chaotic behavior. • Models explaining most neuropsychological syndromes exist; computational psychiatry is rapidly developing since 1995. • Brain-like computing models provide real brain-like functions.=> Complexity of the brain is not the main problem!

17. Chinese room objection Systems that pass Turing test still do not understand the meaning! The men inside follows the rules but does not understand a word – syntactic relations are not sufficient for semantics (J. Searl 1980). Called “arguably the 20th century's greatest philosophical polarizer”, this thought experiment has led to hundreds of articles and books!

18. Solution to the Chinese room • It is not a test – the outcome is always negative! If I go into your head I will not understand either. • Conditions under which human observer could recognize that a system understands should be discussed – a “resonance” of minds. • A feeling “I understand” is confused here with real operational understanding. Some drugs or mental practices induce the illusion of understanding everything; sometimes we have no feeling of understanding, but can answer correctly and in fact do understand. • Searl concludes (wrongly): we know that humans understand, therefore their neurons must have some mysterious causal powers that computer elements do not have. • Correct conclusion: Turing tests is still important, Chinese room fails. This is a trap! Once you treat it seriously it is hard to get out.

19. Hard problem of consciousness Old mind-body problem in new disguise, presented in the Journal of Consciousness Studies in 1995, and in a book Chalmers D.J, The Conscious Mind: In Search of a Fundamental Theory, Oxford University Press 1996 (got > 50 reviews!) • Easy problems: directing attention, recognizing, commenting, etc. • Hard problem of consciousness: qualitative character of phenomenal experience (PE), or qualia – why are we not zombies? Theoretically all information processing could go on without any experience – sweetness of chocolate, or redness of sunset. Qualia = |Conscious perception – Information processing| Inner experience cannot be explained in words, robots can work without it. How to program something that does not make a difference?

20. Hard problem solution A lot of nonsense has been written on qualia. Some solutions: there is no problem; we will never solve it; information processing has dual aspects, physical and phenomenal; panpsychism; protophenomena; quantum C ... • 8 years of discussions led nowhere. A fruitful way proposed by Thomas Reid (1785), and Indian philosophers 2000 years before him, distinguishes clearly between sensation (feeling) and perception (judgment, discrimination). I feel pain: makes an impression that some 'I' has an object 'pain'. Reification of the process into an object creates a mystery. It is just 'pain', sensation, a process, activity, system response. Red color has a particular feeling to it: sure! It corresponds to real, specific brain states/processes that differ from brain states associated with other perceptions.

21. But why do qualia exist? Imagine a rat smelling food. In fraction of a second rat has to decide: eat or spit? • Smell and taste a bit. • Request for comments is send to memory from the gustatory cortex. • Memory is distributed, all brain has to be searched for associations. • Request appears as a working memory (WM) pattern at the global brain dynamics level. • WM is small, just a few patterns fit in (about 7 in humans). • Resonant states are formed activating relevant memory traces. • Answer appears: bad associations! probably poison! spit! • Strong physiological reaction starts – perception serves action. • The WM episodic state is stored for future reference in LTM. • Rat has different "feelings" for different tastes. If the rat could comment on such episode, what would it say? • Results of this non-symbolic, continuous taste discrimination have to be remembered and associated with some reactions: qualia!

22. More on qualia Long Term Memory (LTM) is huge, stored by 100T synapses. Working Memory (WM) is probably based on dynamical brain states (actualization of LTM potential possibilities). • Adaptive resonant states: the up-going (sensory=>conceptual) and the down-going (conceptual=>sensory) streams of information self-organize to form reverberations, transitory brain/mind states. • Resonant states are “dressed”: they contain associations, memories, motor or action components, in one dynamical flow – this is quite differentfrom abstract states of the Turing machine registers. What happens to the taste of a large ice-cream? The taste buds provide all the information; the brain processes it, but the qualia are gone after a short time. Why? WM is filled with other objects, no resonances with gustatory cortex are formed, no reference to taste memories.

23. Automatization of actions Learning: initially conscious involvement (large brain areas active) in the end becomes automatic, subconscious, intuitive (well-localized activity). Formation of new resonant states - attractors in brain dynamics during learning => neural models. Reinforcement learning requires observing and evaluating how successful are the actions that the brain has planned and is executing. Relating current performance to memorized episodes of performance requires evaluation + comparison (Gray – subiculum), followed by emotional reactions that provide reinforcement via dopamine release, facilitating rapid learning of specialized neural modules. Working memory is essential to perform such complex task.Errors are painfully conscious, and should be remembered. Conscious experiences provide reinforcement; there is no transfer from conscious to subconscious.

24. Why do we feel the way we do? Qualia must exist in brain-like computing systems: • Qualia depend on cognitive mechanisms; habituation, intensive concentration or attention may remove qualia. • Qualia require correct interpretation, ex: segmentation of visual stimuli from the background; no interpretation = no qualia. • Secondary sensory cortex is responsible for interpretation; lesions will lead to change in qualia (asymbolia). • Visual qualia: clear separation between higher visual areas (concepts, object recognition) and lower visual areas; activity of lower only should lead to qualia (eg. freezing V4 - no color qualia). • Memory is involved in cognitive interpretation: qualia are altered by drugs modifying memory access. • Cognitive training enhances all sensory qualia; memorization of new sounds/tastes/visual objects changes our qualia. • New qualia are also accessible in dreams. • How does it feel to do the shoe laces? Episodic memory (resonant states) leads to qualia; procedural memory (maps) - no qualia. • Phenomenology of pain: no pain without cognitive interpretation. • Wrong interpretation of brain states – unilateral neglect, body dysmorphia, phantom limbs controlled by visual stimulation mirrors. • Blindsight, synesthesia, absorption states ... many others.

25. Requirements for qualia System capable of evaluation of their WM states, must claim to have phenomenal experiences and be conscious of these experiences! Minimal conditions for an artilect to claim qualia and be conscious: • Working Memory (WM), a recurrent dynamic model of current global system (brain) state, containing enough information to re-instate the dynamical states of all the subsystems. • Permanent memory for storing pointers that re-instate WM states. • Ability to discriminate between continuously changing states of WM; "discrimination" implies association with different types of responses or subsequent states. • Mechanism for activation of associations stored in permanent memory and for updating WM states. • Act or report on the actual state of WM. • Representation of 'the self', categorizing the value of different states from the point of view of the goals of the system, which are implemented as drives, giving a general orientation to the system.

26. Brain-like computing • I can see, hear and feel only my brain states! Ex: change blindness. • Cognitive processes operate on highly processed sensory data. • Redness, sweetness, itching, pain ... are all physical states of brain tissue. Brain states are physical, spatio-temporal states of neural tissue. In contrast to computer registers, brain states are dynamical, and thus contain in themselves many associations, relations. Inner world is real! Mind is based on relations of brain’s states. Computers and robots do not have an equivalent of such WM.

27. Towards conscious robots Few explicit attempts to build them so far. Stan Franklin, "Conscious" Software Research Group, Institute of Intelligent Systems, University of Memphis, CMattie project: an attempt to design and implement an intelligent agent under the framework of Bernard Baars' Global Workspace Theory. Owen Holland, University of Essex: consciousness via increasingly intelligent behavior, robots with internal models, development of complex control systems, looking for “signs of consciousness”, 0.5 M£ grant. Pentti Haikonen (Nokia, Helsinki), The cognitive approach to conscious machines (Imprint Academic 2003). Simulations + microchips coming. More links at: http://www.phys.uni.torun.pl/~duch/ai-ml.html Do we want to have conscious robots? Perhaps yes.

28. Typical design Haikonen has done some simulations based on a rather straightforward design, with neural models feeding the sensory information (with WTA associative memory) into the associative “working memory” circuits.

29. Conclusions Robots and avatars will make a steady progress towards realistic human-like behavior – think about progress in computer graphics. • Artificial minds of brain-like systems will claim qualia; they will be as real in artificial systems as they are in our brains. • There are no good arguments against convergence of the neural modeling process to conscious artifacts. • Achieving human-level competence in perception, language and problem-solving may take longer than creation of basic consciousness. Creation of conscious artilects will open Pandora’s box What should be their status? Will it degrade our own dignity?Is switching off a conscious robot a form of killing?...Will they ever turn against us ...

30. ??? Links to bots and more are at: http://www.ntu.edu.sg/home/aswduch/ai-ml.html#Agent This talk: http://www.ntu.edu.sg/home/aswduch/ref/03-Brain-like/Bots+MC.ppt