Pat Langley Seth Rogers Computational Learning Laboratory Center for the Study of Language and Information Stanford Univ

1. An Extended Theory of Human Problem Solving Pat Langley Seth Rogers Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA http://cll.stanford.edu/ Thanks to D. Choi, K. Cummings, N. Nejati, S. Sage, D. Shapiro, and J. Xuan for their contributions. This talk reports research. funded by grants from DARPA IPTO and the US National Science Foundation, which are not responsible for its contents.

2. The Standard Theory of Problem Solving Traditional theories claim that human problem solving occurs in response to unfamiliar tasks and involves: the mental inspection and manipulation of list structures; search through a space of states generated by operators; backward chaining from goals through means-ends analysis; a shift from backward to forward chaining with experience. These claims characterize problem solving accurately, but this does not mean they are complete.

3. Further Claims about Problem Solving We maintain that the standard theory is incomplete and that: Human problem solving occurs in a physical context. Problem solving abstracts away from physical details, yet must return to them for implementing solutions. Problem solving interleaves reasoning with execution. Eager execution can lead to situations that require restarts. Learning from solutions transforms backward chaining into informed forward execution. These claims are not entirely new, but they have received little attention in previous computational models.

4. The ICARUS Architecture We have embedded these extensions in ICARUS, a cognitive architecture that builds on five principles: Cognition grounded in perception and action Cognitive separation of categories and skills Hierarchical organization of long-term memory Cumulative learning of skill/concept hierarchies Correspondence of long-term/short-term structures These ideas distinguish ICARUS from other architectures like ACT-R, Soar, and EPIC.

5. Hierarchical Structure of Long-Term Memory ICARUS organizes both concepts and skills in a hierarchical manner. concepts Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. skills

6. Perceptual Buffer ICARUS’ Memories and Processes Short-Term Conceptual Memory Long-Term Conceptual Memory Categorization and Inference Perception Environment Skill Retrieval Long-Term Skill Memory Short-Term Goal/Skill Memory Problem Solving Skill Learning Skill Execution Motor Buffer

7. The Physical Context of Problem Solving ICARUS is a cognitive architecture for physical, embodied agents. On each successive perception-execution cycle, the architecture: places descriptions of sensed objects in the perceptual buffer; infers instances of concepts implied by the current situation; finds paths through the skill hierarchy from top-level goals; selects one or more applicable skill paths for execution; invokes the actions associated with each selected path. Problem solving in ICARUS builds upon this basic ability to recognize physical situations and execute skills therein.

8. Basic ICARUS Processes ICARUS matches patterns to recognize concepts and select skills. concepts Concepts are matched bottom up, starting from percepts. Skill paths are matched top down, starting from intentions. skills

9. Abstraction from Physical Details ICARUS typically pursues problem solving at an abstract level: conceptual inference augments perceptions using high-level concepts that provide abstract state descriptions. execution operates over high-level durative skills that serve as abstract problem-space operators. both inference and execution occur in an automated manner that demands few attentional resources. However, concepts are always grounded in primitive percepts and skills always terminate in executable actions. ICARUS holds that cognition relies on a symbolic physical system which utilizes mental models of the environment.

10. Interleaved Problem Solving and Execution ICARUS includes a module for means-ends problem solving that: chains backward off skills that would produce the goal; chains backwards off concepts if no skills are available; creates subgoals based on skill or concept conditions; pushes these subgoals onto a goal stack and recurses; executes any selected skill as soon as it is applicable. Embedding execution within problem solving reduces memory load and uses the environment as an external store.

11. Restarting on Problems Even when combined with backtracking, eager execution can lead problem solving to unrecoverable states. ICARUS’ problem solver handles such untenable situations by: detecting when action has made backtracking impossible; storing the goal context to avoid repeating the error; physically restarting the problem in the initial situation; repeating this process until succeeding or giving up. This strategy produces quite different behavior from the purely mental systematic search assumed by most models.

12. Solve(G) Push the goal literal G onto the empty goal stack GS. On each cycle, If the top goal G of the goal stack GS is satisfied, Then pop GS. Else if the goal stack GS does not exceed the depth limit, Let S be the skill instances whose heads unify with G. If any applicable skill paths start from an instance in S, Then select one of these paths and execute it. Else let M be the set of primitive skill instances that have not already failed in which G is an effect. If the set M is nonempty, Then select a skill instance Q from M. Push the start condition C of Q onto goal stack GS. Else if G is a complex concept with the unsatisfied subconcepts H and with satisfied subconcepts F, Then if there is a subconcept I in H that has not yet failed, Then push I onto the goal stack GS. Else pop G from the goal stack GS and store information about failure with G's parent. Else pop G from the goal stack GS. Store information about failure with G's parent. Interleaved Problem Solving and Execution This is traditional means-ends analysis, with three exceptions: (1) conjunctive goals must be defined concepts; (2) backward chaining occurs over both skills and concepts; and (3) selected skills are executed whenever applicable.

13. Learning from Problem Solutions ICARUS incorporates a mechanism for learning new skills that: operates whenever problem solving overcomes an impasse; incorporates only information available from the goal stack; generalizes beyond the specific objects concerned; depends on whether chaining involved skills or concepts; supports cumulative learning and within-problem transfer. This skill creation process is fully interleaved with means-ends analysis and execution. Learned skills carry out forward execution in the environment rather than backward chaining in the mind.

14. Skill Hierarchy Problem Primitive Skills Executed plan ? Execution, Problem Solving, and Learning Skill Execution no impasse? yes Problem Solving Skill Learning

15. C B B A A C Constructing Skills from a Trace (clear C) 4 3 1 (hand-empty) (unst. C B) (clear B) (unstack C B) (on C B) (unst. B A) (clear A) (unstack B A) 2 (ontable A T) (holding C) (hand-empty) (putdown C T) (on B A) (holding B)

16. (clear (?C) :percepts ((block ?D) (block ?C)) :start (unstackable ?D ?C) :skills ((unstack ?D ?C)))(clear (?B) :percepts ((block ?C) (block ?B)) :start [(on ?C ?B) (hand-empty)] :skills ((unstackable ?C ?B) (unstack ?C ?B)))(unstackable (?C ?B) :percepts ((block ?B) (block ?C)) :start [(on ?C ?B) (hand-empty)] :skills ((clear ?C) (hand-empty)))(hand-empty ( ) :percepts ((block ?D) (table ?T1)) :start (putdownable ?D ?T1) :skills ((putdown ?D ?T1))) Learned Skills in the Blocks World

17. Three Questions about Skill Learning What is the hierarchical structure of the skill network? The structure is determined by the subproblems that arise in problem solving, which, because operator conditions and goals are single literals, form a semilattice. What are the heads of the learned clauses/methods? The head of a learned clause is the goal literal that the planner achieved for the subproblem that produced it. What are the conditions on the learned clauses/methods? If the subproblem involved skill chaining, they are the conditions of the first subskill clause. If the subproblem involved concept chaining, they are the subconcepts that held at the outset of the subproblem.

18. Related Theoretical Extensions We are not the first to propose revisions to the standard theory: Zhang and Norman have noted the role of external memory; Gunzelman has modeled interleaved planning and execution; Jones and Langley modeled restarts on unsolved problems; Soar and Prodigy learn production rules from impasses. However, ICARUS is the first cognitive architecture that includes these extensions in a unified way.

19. Many questions about human problem solving still remain open: Directions for Future Research How much do humans abstract away from physical details? How often do they return to this setting during their search? How tightly do they interleave cognition with execution? Under what conditions do they start over on a problem? How rapidly do they acquire automatized strategies? We should address these and related issues in future extensions to the standard theory of problem solving.