Cognitive Science and its Applications

1. Cognitive Science and its Applications R. Karthikeyan, Associate Professor/CSE PSNACET

2. Cognitive • Pertaining to the act or process of knowing, perceiving, remembering etc. • Pertaining to the mental processes of perception, memory, judgment and reasoning

3. Cognitive Science • Interdisciplinary scientific study of the mind and its processes • Examines what cognition is, what it does and how it works. • Includes research on intelligence and behavior, especially focusing on how information is represented, processed, and transformed within nervous systems and machines • Multiple research disciplines – Psychology, AI, Philosophy, Neuroscience, linguistics, and Anthropology.

4. Psychology Cognitive science Social psychology Educational psychology

5. Levels of analysis • Computational theory, specifying the goals of the computation • Representation and algorithms, giving a representation of the inputs and outputs and the algorithms which transform one into the other • Hardware implementation, how algorithm and representation may be physically realized.

6. Example • Task: Multiplication.Input numbers x and y. Output x times y. • Algorithm: Given inputs x and y, look up number z on row x and column y. Output z. • Implementation: human being and paper.

7. Alternative algorithm • How to calculate x times y: • If x =1, then answer is y. • If x>1, add y to itself (x-1) number of times.The result is the answer. • Example: 3x5 = 5+5+5 = 15

8. Two points • The same task can be performed with different algorithms. • Two different systems can do the same task in very different ways. • The same algorithm can be implemented with different hardware.

9. Scope • “Cognitive“ is used for "any kind of mental operation or structure that can be studied in precise terms“ • Large field and covers a wide array of topics on cognition • Social and cultural factors • emotion, consciousness  • Animal cognition, comparative and evolutionary approaches

10. Artificial intelligence • Major contribution of AI and cognitive science to psychology - information processing model of human thinking • Involves the study of cognitive phenomena in machines. • Computers are also widely used as a tool with which to study cognitive phenomena.  • Computational modeling uses simulations to study how human intelligence may be structured.

11. Information processing everywhere • Perception • acquiring real-time information about the surrounding environment. • Language use • making use of information about syntax, semantics and phonology. • Reasoning • combining different sources of information, deriving new information, testing consistency of information, etc. • Action • making use of information in action planning and guidance. • Memory • storing and retrieving information

12. Attention • Selection of important information • The human mind is bombarded with millions of stimuli and it must have a way of deciding which of this information to process. • Experiments that support this metaphor include the dichotic listening task and studies of inattentional blindness.

13. Attention(Contd.) • In the dichotic listening task, subjects are bombarded with two different messages, one in each ear, and told to focus on only one of the messages. • At the end of the experiment, when asked about the content of the unattended message, subjects cannot report it.

14. Knowledge and processing of language • Ability to learn and understand language is an extremely complex process • Driving research questions in studying how the brain itself processes language • To what extent is linguistic knowledge innate or learned? • Why is it more difficult for adults to acquire a second-language than it is for infants to acquire their first-language? • How are humans able to understand novel sentences?

15. VP VP NP V NP PP V discuss N PP discuss N P NP violence P NP violence on TV on TV Knowledge and processing of language(contd.) • Example: Syntactic Disambiguation • “We shall discuss violence on TV.” • Two interpretations :

16. Learning and Development • Learning and development are the processes by which we acquire knowledge and information over time. • Infants are born with little or no knowledge (depending on how knowledge is defined), yet they rapidly acquire the ability to use language, walk, and recognize people and objects. • Research in learning and development aims to explain the mechanisms by which these processes might take place.

17. Memory • Memory allows us to store information for later retrieval. Memory is often thought of consisting of both a long-term and short-term store. • Long-term memory allows us to store information over prolonged periods (days, weeks, years). We do not yet know the practical limit of long-term memory capacity. • Short-term memory allows us to store information over short time scales (seconds or minutes).

18. Memory(contd.) • Memory is also often grouped into declarative and procedural forms • Declarative memory—grouped into subsets of semantic and episodic forms of memory-refers to our memory for facts and specific knowledge, specific meanings, and specific experiences(e.g. "Who was the first president of the U.S.A.?", or "What did I eat for breakfast four days ago?"). • Procedural memory allows us to remember actions and motor sequences (e.g. how to ride a bicycle) and is often dubbed implicit knowledge or memory

19. Perception and action • Perception is the ability to take in information via the senses, and process it in some way.  • Vision and hearing are two dominant senses that allow us to perceive the environment • Action is taken to refer to the output of a system. In humans, this is accomplished through motor responses. • Spatial planning and movement, speech production, and complex motor movements are all aspects of action.

20. Applications of Cognitive Science

21. Virtual Human- Human Computer Interaction Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them

22. Human • Humans are limited in their capacity to process information. This has important implications for Design • Information is received and responses given via a number of input and output channels: • Visual Channel • Auditory Channel • Haptic Channel • Movement • Information is stored in memory: • Sensory Memory • Working Memory • Long-Term Memory

23. Computer • There is not much difference in Human and Computer • Computer consist of • Input Devices • Output Devices • Memory • Processing • Computer can be Mobile, Spacecraft Cockpit, etc.

24. Interaction • HCI is concerned with joint performance of Task by Human & Computer • Communication between The User and The System • Physical Interaction • Interaction Devices • Conceptual Interaction • Interaction Styles

25. Goals • The main goal of HCI is Usability • “It is a measure of the effectiveness, efficiency and satisfaction with which specified user can achieve specified goals in a particular environment” • A usable System is: • Easy to Learn • Easy to Remember how to Use • Effective to Use • Efficient to Use • Safe to Use • Enjoyable To Use

26. The Model Human Processor • The model human processor consists of three interacting systems. Each has its own memory and processor. • Perceptual processor • Outputs into audio storage • Outputs into visual storage • Cognitive processor • Outputs into working memory. • Has access to: Working memory Long term memoryMotor processor • Carries out actions

27. User Interaction Design • Data Collection • Data Analysis • User Modeling • Design • Prototyping • Evaluation

28. Identify needs/establish requirements Evaluate (Re)Design Build an interactive version Final product User Interaction Design Flow

29. Interaction Styles • Command Line Interface • Menus • Question & Answer Dialogue • Form-Fill Interface • WIMP • Direct Manipulation • Natural Language

30. Interaction Devices • Input Devices • Keyboards • Mouse, Trackballs, Touch Pads • Touch Screens • Speech Input • 3D trackers, 3D Mouse • Output Devices • Screens • Printers • Sound Output

31. Layered Virtual Human

32. Layered Virtual Human(contd.) • Cognitive Layer Inner layer where cognitive components exist One cognitive level per virtual human Mind of the virtual human that makes decisions based on input, goals, and desired behavior. • Virtual Human Layer Consists of components that make up the virtual human Input : Vision, Speech and Smell Output : Verbal speech, Body gestures, Actions This layer can be used by one or more virtual human. Closely connected to cognitive layer .

33. Layered Virtual Human(contd.) • Simulation Layer This is everything else that has to do with the environment that the virtual humans exist in. Include the game engine that creates the world that the characters are displayed in and interact with a world. All input from the real world, like gesture recognition, object or human positions, microphones or cameras would feed into the simulation layer.

34. Sense-Think-Act Cycle

35. Sense-Think-Act Cycle(contd.) • Virtual humans follow the same paradigm as Belief- Desire-Intention (BDI) style agents with a sense-think- act cycle • The cognitive and virtual human layers correspond to the right side of this diagram • Simulation and real world correspond to the left side • The virtual human research attempts to integrate all of these layers and attempts to answer some fundamental questions about how they should be built, and how they can be effectively used in training environments.

36. Virtual Human Integrated Architecture

37. Virtual Human Integrated Architecture(contd) • Cognitive Layer Intelligent Agent - major reasoning engine of the agent, based on the SOAR Cognitive Architecture. SOAR is a symbolic reasoning system that includes concepts such as long term and short term memory, goal directed behavior and a decision procedure for selecting rules. Dialog Manager – reason about trainee’s utterances Emotional Modeling - appraising current events and the current situation and comparing them with the agent's beliefs and goals to determine emotional state and appropriate coping behavior.

38. Virtual Human Integrated Architecture(contd) • Virtual Human Layer Speech recognition Natural Language Understanding Natural Language Generation Non verbal Behavioral Generator Smart Body Text to Speech

39. Virtual Human Integrated Architecture(contd) • Simulation Layer Unreal Tournament and Integrated Architecture Gaze/Gesture Tracking/Input system

40. SASO-ST [Stability and Support Operations - Simulation and Training]

41. SASO-ST • A research prototype demonstrating advanced virtual human technologies in a new negotiation domain. • Trainees are to communicate in real-time with an embodied virtual human doctor to negotiate and convince Dr. Perez to move the clinic out of harms way

42. ELECT-Bilat

43. ELECT-Bilat • Enhanced Learning Environment with Creative Technologies is a game based simulation for soldiers to practice and conduct bilateral engagements in a cultural context that includes virtual humans that verbally respond to the selected questions. • Uses a menu based system instead of speech recognition

44. Cognition reigns but does not rule Paul Valéry

45. Thank You