Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA

1. Varieties of Problem Solving in a Unified Cognitive Architecture Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Thanks to D. Choi, T. Konik, U. Kutur, D. Nau, S. Rogers, and D. Shapiro for their many contributions. This talk reports research funded by grants from DARPA IPTO, which is not responsible for its contents.

2. The ICARUS Architecture ICARUS is a theory of the human cognitive architecture that posits: Short-term memories are distinct from long-term stores Memories contain modular elements cast as symbolic structures Long-term structures are accessed through pattern matching Cognition occurs in retrieval/selection/action cycles Learning involves monotonic addition of elements to memory Learning is incremental and interleaved with performance It shares the assumptions with other cognitive architectures like Soar (Laird et al., 1987) and ACT-R (Anderson, 1993).

3. Distinctive Features of ICARUS However, ICARUSalso makes assumptions that distinguish it from these architectures: Cognition is grounded in perception and action Categories and skills are separate cognitive entities Short-term elements are instances of long-term structures Inference and execution are more basic than problem solving Skill/concept hierarchies are learned in a cumulative manner Some of these tenets also appear in Bonasso et al.’s (2003) 3T, Freed’s APEX, and Sun et al.’s (2001) CLARION.

4. Cascaded Integration in ICARUS Like other unified cognitive architectures, ICARUSincorporates a number of distinct modules. learning problem solving skill execution conceptual inference ICARUSadopts a cascaded approach to integration in which lower-level modules produce results for higher-level ones.

5. Perceptual Buffer ICARUS’ Memories and Processes Short-Term Belief Memory Long-Term Conceptual Memory Conceptual Inference Perception Environment Skill Retrieval and Selection Long-Term Skill Memory Short-Term Goal Memory Problem Solving Skill Learning Skill Execution Motor Buffer

6. Hierarchical Structure of Memory ICARUS interleaves its long-term memories for concepts and skills. concepts Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. skills

7. Hierarchical Structure of Memory ICARUS interleaves its long-term memories for concepts and skills. concepts Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. skills

8. Basic ICARUS Processes ICARUS matches patterns to recognize concepts and select skills. concepts Concepts are matched bottom up, starting from percepts. Skill paths are matched top down, starting from intentions. skills

9. ICARUS Interleaves Execution and Problem Solving Skill Hierarchy Problem Reactive Execution ? no impasse? Primitive Skills yes Executed plan Problem Solving This organization reflects the psychological distinction between automatized and controlled behavior.

10. Previous versions of ICARUS have used means-ends analysis, which has been observed repeatedly in humans, but it differs from most versions in that it interleaves backward chaining with execution. Means-Ends Problem Solving in ICARUS Solve(G) Push the goal literal G onto the empty goal stack GS. On each cycle, If the top goal G of the goal stack GS is satisfied, Then pop GS. Else if the goal stack GS does not exceed the depth limit, Let S be the skill instances whose heads unify with G. If any applicable skill paths start from an instance in S, Then select one of these paths and execute it. Else let M be the set of primitive skill instances that have not already failed in which G is an effect. If the set M is nonempty, Then select a skill instance Q from M. Push the start condition C of Q onto goal stack GS. Else if G is a complex concept with the unsatisfied subconcepts H and with satisfied subconcepts F, Then if there is a subconcept I in H that has not yet failed, Then push I onto the goal stack GS. Else pop G from the goal stack GS and store information about failure with G's parent. Else pop G from the goal stack GS. Store information about failure with G's parent.

11. C B B A A C A Successful Means-Ends Trace initial state (clear C) (hand-empty) (unst. C B) (clear B) (unstack C B) goal (on C B) (unst. B A) (clear A) (unstack B A) (ontable A T) (holding C) (hand-empty) (putdown C T) (on B A) (holding B)

12. However, in some domains, humans carry out forward-chaining search with methods like progressive deepening (de Groot, 1978). In response, we have added a new module to ICARUS that: Problem Solving as Iterative Sampling • performs mental simulation of a single trajectory consistent with its stored hierarchical skills; • repeats this process to find a number of alternative paths from the same initial state; • selects the path that produces the best outcome to determine the next primitive skill to execute. We refer to this memory-limited search method as hierarchical iterative sampling (Langley, 1992).

13. Our initial version of forward search makes a few implausible psychological assumptions: More on Iterative Sampling • stochastic path selection and final choices are based on “reachability heuristics” from the AI planning literature; • parameters determine the depth of search and number of iterations, rather than memory capacity and time available; • no progressive deepening occurs when two alternatives produce similar scores. Nevertheless, it seems a promising first step toward modeling heuristic search in domains like chess.

14. A key question concerns when humans carry out means-ends analysis vs. forward search; some candidate hypotheses are: Unifying Forward and Backward Search • they use backward chaining except when the branching factor from the goal becomes too large, as in most games; • they favor backward chaining when goals are very specific and forward search for less constrained goals; • they prefer backward chaining but fall back on forward search when they retrieve no relevant skills (Jones & Langley, 2006). We need detailed psychological studies to select among these alternatives or replace them with better ones. Once answered, we can incorporate the results into ICARUS to offer a unified theory of human problem solving.

15. ICARUS is a unified theory of the cognitive architecture that: Contributions of the Research includes hierarchical memories for concepts and skills; interleaves conceptual inference with reactive execution; resorts to problem solving when it lacks relevant skills; carries out both means-ends analysis and forward search. The latter each account for some aspects of human problem solving, but not for when to invoke each method. Explaining this choice should be a high priority for future work. For more information about the ICARUS architecture, see: http://cll.stanford.edu/research/ongoing/icarus/

16. End of Presentation

17. ICARUS Concepts for In-City Driving ((in-rightmost-lane ?self ?clane) :percepts ( (self ?self) (segment ?seg) (line ?clane segment ?seg)) :relations ((driving-well-in-segment ?self ?seg ?clane) (last-lane ?clane) (not (lane-to-right ?clane ?anylane)))) ((driving-well-in-segment ?self ?seg ?lane) :percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)) :relations ((in-segment ?self ?seg) (in-lane ?self ?lane) (aligned-with-lane-in-segment ?self ?seg ?lane) (centered-in-lane ?self ?seg ?lane) (steering-wheel-straight ?self))) ((in-lane ?self ?lane) :percepts ( (self ?self segment ?seg) (line ?lane segment ?seg dist ?dist)) :tests ( (> ?dist -10) (<= ?dist 0)))

18. Representing Short-Term Beliefs/Goals (current-street me A) (current-segment me g550) (lane-to-right g599 g601) (first-lane g599) (last-lane g599) (last-lane g601) (at-speed-for-u-turn me) (slow-for-right-turn me) (steering-wheel-not-straight me) (centered-in-lane me g550 g599) (in-lane me g599) (in-segment me g550) (on-right-side-in-segment me) (intersection-behind g550 g522) (building-on-left g288) (building-on-left g425) (building-on-left g427) (building-on-left g429) (building-on-left g431) (building-on-left g433) (building-on-right g287) (building-on-right g279) (increasing-direction me) (buildings-on-right g287 g279)

19. ICARUS Skills for In-City Driving ((in-rightmost-lane ?self ?line) :percepts((self ?self) (line ?line)) :start ((last-lane ?line)) :subgoals ((driving-well-in-segment ?self ?seg ?line))) ((driving-well-in-segment ?self ?seg ?line) :percepts((segment ?seg) (line ?line) (self ?self)) :start ((steering-wheel-straight ?self)) :subgoals ((in-segment ?self ?seg) (centered-in-lane ?self ?seg ?line) (aligned-with-lane-in-segment ?self ?seg ?line) (steering-wheel-straight ?self))) ((in-segment ?self ?endsg) :percepts((self ?self speed ?speed) (intersection ?int cross ?cross) (segment ?endsg street ?cross angle ?angle)) :start ((in-intersection-for-right-turn ?self ?int)) :actions((steer 1)))

20. Directions for Future Research Future work on ICARUS should incorporate other ideas about: progressive deepening in forward-chaining search graded nature of categories and category learning model-based character of human reasoning persistent but limited nature of short-term memories creating perceptual chunks to reduce these limitations storing and retrieving episodic memory traces These additions will increase further ICARUS’ debt to psychology. For more details, see:http://cll.stanford.edu/research/ongoing/icarus/