An Introduction to CLARION

1. An Introduction to CLARION • Connectionist Learning with Adaptive Rule Induction ON-line) is a recent cognitive architecture suitable for social simulation. It has four functional subsystems: the Action-Centered Subsystem (ACS), the Non-Action-Centered Subsystem (NACS), the Meta-Cognitive Subsystem (MCS), and the Motivational Subsystem (MS). • Ref. Ron Sun, 2004, The CLARION cognitive architecture: extending cognitive modeling to social simulation. . U S Tiwary, Indian Institute of Information Technology , Allahabad

2. CLARION Architecture U S Tiwary, Indian Institute of Information Technology , Allahabad

3. The Architecture: ACS • ACS : The action centered subsystem is mainly used for action decision making. The implicit and explicit processes make their separate decision making; their results are then combined to decide the action. If the resulting action is successful, then a rule is extracted at top level. Based on subsequent interaction this rule will be either generalized or made more specific. U S Tiwary, Indian Institute of Information Technology , Allahabad

4. The Architecture: NACS, MS & MCS • NACS : The NACS maintains general world knowledge, both implicit and explicit, and is responsible for reasoning about the world. • The MS and MCS supervise the operations of ACS and NACS. • The MS provides the context in which the goal and the reinforcement of ACS are determined. • The main task of MCS is cognitive monitoring and parameter setting of ACS and NACS. It consists of different types of meta-cognitive processes to achieve its functionalities. U S Tiwary, Indian Institute of Information Technology , Allahabad

5. Main Characteristics(1) • Use of dual representational structure: Each subsystem in CLARION uses separate representation for implicit and explicit knowledge. Each subsystem uses dual representational structure. The top level corresponds to explicit knowledge (easily accessible) whereas the bottom level corresponds to implicit knowledge (less accessible and more “holistic). U S Tiwary, Indian Institute of Information Technology , Allahabad

6. Main Characteristics(2) • Top-down and bottom-up learning: An agent in CLARION can learn on its own, i. e., in the absence of externally supplied knowledge, explicit knowledge may be developed gradually through a trial and error basis. Top-down learning is also supported in CLARION. • This is unlike ACT-R, which focuses mainly on top-down learning. U S Tiwary, Indian Institute of Information Technology , Allahabad

7. Main Characteristics(3) • Cognitive-Meta-Cognitive Interaction: The subsystems in this architecture interact constantly to accomplish cognitive processing. This interaction may be executive control of some sub-system or meta-cognitive control and monitoring of some process. • Such interaction is not fully supported in SOAR and ACT-R. U S Tiwary, Indian Institute of Information Technology , Allahabad

8. Main Characteristics(4) • Use of motivational structures: CLARION includes motivational structures which help agents in understanding and appreciating each others’ motivation and find ways to cooperate. This helps in social interaction U S Tiwary, Indian Institute of Information Technology , Allahabad

9. Details of Char…Dual Representation • Dichotomy and Dual Representation : - Some well-known dichotomies in cognitive science : • Symbolic Vs Subsymbolic processing • Conceptual Vs Subconceptual processing • Dichotomies are well studied in Psychological studies, including in social psychology, of Implicit and Explicit Cognition (Learning, Memory and Perception) • These studies justify the notion of implicit and explicit cognition , which is the focus of CLARION. U S Tiwary, Indian Institute of Information Technology , Allahabad

10. Char…Motivational Subsystem • Motivations of an agent is prior to Cognition, but Cognition basically evolves to serve the essential needs and motivations of an agent. • In the process, Cognition takes into account the environments (physical or social) , their regularities and structures • Some needs and motivations are inherently social or socially oriented U S Tiwary, Indian Institute of Information Technology , Allahabad

11. Char…Implicit and explicit and top-down bottom up learning • Agent may learn on its own, whether or not there is a priori oe externally provided domain knowledge. • Learning may proceed on a trial and error basis. • Through a bottom-up learning process , explicit and abstract domain knowledge may be developed, in gradual and incremental fashion.. • The arch also provides innate biases and innate behavioural propensities within the arch ,, in terms of constraining, guiding and facilitating learning through top down learning. • Top down learning can provide assimilation of explicit/abstract knowledge from external sources into implicit forms. U S Tiwary, Indian Institute of Information Technology , Allahabad

12. Char… Multiple Subsystems • Multiple subsystems interact constantly for cognitive processing. • The interaction may include some ‘executive control’ of some subsystems or metacognitive monitoring of some ongoing processing • The metacognitive subsystem can self monitor, reflect and dynamically modify its own behaviour U S Tiwary, Indian Institute of Information Technology , Allahabad

13. Multiple Subsystems Interaction for Social Interaction • Unlike other architectures, interaction between motivational structures and other subsystems is important for social interaction. • Agents can understand and appreciate each other’s innate or acquired motivational structures and can find ways to cooperate. U S Tiwary, Indian Institute of Information Technology , Allahabad

14. Algorithm for Action Decision Making 1. Observe the current state x. 2. Compute in the bottom level (the IDNs) the value of each of the possible actions (ai s) associated with the state x : ( Q(x,a1), Q(x,a2),………Q(x,an)). Stochastically choose one action according to these values. 3. Find out all possible actions (b1, b2, … bm) at the top level, based on the current state x ( which goes up from the bottom level) and the existing rules in place at the top level. Stochastically choose one action. U S Tiwary, Indian Institute of Information Technology , Allahabad

15. Algorithm for Action Decision Making 4. Choose an appropriate action, by stochastically selecting the outcome of either the top level or the bottom level. 5. Perform the action, and observe the next state y and (possibly) the reinforcement r. 6. Update the bottom in accordance with an appropriate algorithm based on the feedback information. 7. Update the top level using an appropriate algorithm ( for extracting, refining, and deleting rules). U S Tiwary, Indian Institute of Information Technology , Allahabad

16. Q-Learning Describe on the blackboard. U S Tiwary, Indian Institute of Information Technology , Allahabad