A Gentle Introduction to Soar, an Architecture for Human Cognition: 2006 Update

1. A Gentle Introduction to Soar,an Architecture for Human Cognition: 2006 Update Jill Fain Lehman, John Laird, Paul Rosenbloom Soft Computing Lab. Yongjun Kim 26th Mar., 2009

2. Outline • Introduction • Soar (State operator and result) Architecture • The Idea of Architecture • What Cognitive Behaviors have in common • Behavior as Movement through Problem Spaces • Tying the Content to the Architecture • Memory, Perception, Action, and Cognition • Detecting a Lack of Knowledge • Learning • Putting it all together: a Soar Model of Joe Rookie • Stepping back: the Soar Architecture in review • From Architecture to Unified Theories of Cognition • Discussion

3. Introduction • Many intellectual disciplines contribute to cognitive science: • Psychology, linguistics, anthropology, artificial intelligence, etc. • Each discipline provides expertise and contributes microtheories. • Descriptions of regularities in behavior • Theories that try to explain those regularities • How to know the contributions of each discipline fit in the big picture? • Go ahead and try to put the whole picture together. • Try to build unified theories of cognition (UTCs). • A set of general assumptions for cognitive models that account for all of cognition. • Soar was developed and used as a candidate UTC in 1980’s. • Try to find a set of computationally-realizable mechanisms and structures that can answer all the questions about cognitive behavior.

4. The Idea of Architecture • Architecture • Architecture is the fixed set of mechanisms and structures. • An architecture stands as a theory of what is common among behavior. • Any complex system can be decomposed into architecture and content. • Architecture requires content to produce behavior. • Cognitive Architecture • A theory of the fixed mechanisms and structures that underlie human cognition. • Soar is used as a cognitive architecture.

5. What Cognitive Behaviors have in common • Soar theory assumes that cognitive behavior has at least the following characteristics: • It is goal-oriented. • It takes place in a rich, complex, detailed environment. • It requires a large amount of knowledge. • It requires the use of symbols and abstractions. • It is flexible, and a function of the environment. • It requires learning from the environment and experience. • Need to explore the architecture in terms of some particular content in order to see how the architecture contributes to behavior. • Have to be goal-oriented about something. • Need a scenario.

6. What Cognitive Behaviors have in common • A Simple Scenario from Baseball.

7. What Cognitive Behaviors have in common • A Simple Scenario from Baseball. • Behave in a goal-oriented manner: • Joe’s goal is to win the game. • A number of subgoals : get the batter out, strike the batter out with a curve ball, etc. • Operate in a rich, complex, detailed environment: • Many relevant aspects of Joe’s environment. • Positions of people, the number of balls and strikes, etc. • Use a large amount of knowledge: • Need to draw on statistics about his own team, his own pitching record, etc. • Behave flexibly as a function of the environment: • Need to respond to his own perceptions of the environment. • Is it windy?, is the batter left-or right-handed? etc. • Use symbols and abstractions: • Need to draw on his previous experience by abstracting away from this day and place. • Learn from the environment and experience: • Joe need to throw Sam a fast ball next time.

8. What Cognitive Behaviors have in common • A Simple Scenario from Baseball. • For Joe to act like a rookie pitcher, many different kinds of knowledge should be given to it. • Need to find some way to represent and process Joe’s knowledge in Soar. • Assume that there is an underlying structure to behavior and knowledge. • This structure provides a means for organizing knowledge as a sequence of decisions through a problem space.

9. Behavior as Movement through Problem Spaces • The Space of Possible Actions for Joe. • Must make his decisions with respect to the situation at the moment. • Support two points of view : a static / a dynamic view of Joe’s life.

10. Behavior as Movement through Problem Spaces • The Abstract Form of a Problem Space. • Consist of states, features, values and operators. • The state is a representation of all the aspects of the situation (internal and external). • Only one state exists at any time and prior states are not directly accessible. f2 : batter status v6 : out

11. Behavior as Movement through Problem Spaces • The Abstract Form of a Problem Space. • Movement could be entirely random. • To keep behavior goal-directed, the succession of operators and the resulting state transformations must be guided by the principle of rationality: • If an agent has knowledge that an operator application will lead to one of its goals then the agent will select that operator. • Tying the Content of Joe’s World to the Soar Architecture. • Map knowledge into goals, states and operators. • Determine what knowledge becomes part of the state and the operators. • Find how to know what an operator application will do. • Find how to know when the goal has been achieved.

12. Tying the Content to the Architecture • Knowledge Representation in the Architecture • Must be domain independent. • What is common across all domains and problems? • In Soar, it is the decomposition of knowledge into goals, problem spaces, states, and operators.

13. Tying the Content to the Architecture • Guidelines for Tying the Domain Content to the Architecture • The domain knowledge of the objects and people in the game (K1) => the features and values in the state • Knowledge of actions (K5 and K7) => operators • Knowledge about objectives (K4) => goals • Knowledge of the rules of the game (K3) + K1 + K4 + K5 + K7 => problem space • The Operator Selection • Operators that share common tests for goals and situations can be considered to be part of the same problem space. • Given the initial state and the goal in the game, the operators will be the various kinds of pitches.

14. Tying the Content to the Architecture • The Effect of Operators • Can be defined in two ways: • Defined by the execution of the operator in the external world using knowledge of physical actions (K7). • Defined by knowledge of abstract events or particular episodes (K2). • Goal Evaluation • Determining that the current state is a desired state relies on knowledge of the rules of the game (K3). • The environment can give signals of success and failure. • Umpire’s judge (“You’re out!”) • The Modification of Goal and Problem Space • Done by augmenting the state with goals and problem spaces. • Ex. Joe’s team is ahead in the fifth inning, but rain is on the horizon => out quickly. • Ex. A member of the opposing team on base => short windup.

15. Memory, Perception, Action, and Cognition • Soar’s Memory • Consist of long-term memory (LTM) and working memory (WM). • LTM has three different types: procedural, semantic, and episodic.

16. Memory, Perception, Action, and Cognition • Memory type • Long-term memory (LTM) • Procedural : knowledge about how and when to do things. • How to ride a bike, how to solve an algebra problem, etc. • Semantic : knowledge consists of facts about the world. • Bicycles have two wheels, a baseball game has nine innings, etc. • Episodic : knowledge consists of things you remember. • The time you fell off your bicycle and scraped your elbow. • LTM is not directly available, but must be searched to find what is relevant to the current situation. • Procedural knowledge is primarily responsible for controlling behavior and maps directly onto operator knowledge. • Working memory (WM) • Knowledge that is most relevant to the current situation. • In Soar, WM is represented as a set of the features and values that make up the current state (and substates). • Can be used to retrieve other knowledge from LTM. • Working memory elements in Soar arise in one of two ways: • Through perception. • Through retrievals from long-term memory.

17. Memory, Perception, Action, and Cognition • Examples of LTM procedural knowledge • There are dependencies between the rules. However, Soar doesn’t recognize them. • Rules are processed by the architecture in a general way.

18. Memory, Perception, Action, and Cognition • The Decision Cycle • Generate behavior out of the content in LTM and WM. • Do its work in five phases: • Input • WM elements are created. • Elaboration • WM elements are matched against the “if” parts of the rules in LTM. • Decision • Decide suggestions according to preferences (symbolic/numeric). • Application • Output • Support limited parallelism. • Multiple actions can be packaged together as a single operator.

19. Detecting a Lack of Knowledge • Impasse in the Decision Cycle • Happen when the decision cycle can’t decide a single operator due to lack of knowledge for preferences (e.g., without r5). • Soar automatically creates a substate. • The goal is to select between two operators for the original state. • Semantic and episodic memories are usually used in substates. • The reminding is goal-driven.

20. Detecting a Lack of Knowledge • Memory Search in Impasse • Assume that Joe has the following fact in episodic memory. • A Cue must be created that can be used to search the memory. • In some cases, no likely match will be returned. Then, the model can modify the cue. • The next three rules define the evaluate operator that creates preferences to resolve the tie.

21. Detecting a Lack of Knowledge • Resolving an Operator-Tie Impasse

22. Detecting a Lack of Knowledge • Working Memory Hierarchy • Working memory consists of a state/substate hierarchy. • The hierarchy grows as impasses arise and shrinks as impasses are resolved. • If multiple changes are suggested in different states, the change to the state that is highest in the hierarchy is selected. • If a change occurs to a context high up in the hierarchy, then all the substatesbelow the changed state disappear. • Impasse Type in Soar • Other types of impasses can be occurred. • E.g., an operator tie, an operator no-change impasse, etc. • The full set of impasses defined in Soar is fixed and domain-independent.

23. Learning • Four Learning Mechanisms • Chunking • Reinforcement learning • Episodic learning • Semantic learning • Three Questions for Learning Systems: • What do they learn? • Soar systems learn structures for its LTM: rules, declarative facts, and episodes. • What is the source of knowledge for learning? • Different between the learning mechanisms. • When do they learn? • Different between the learning mechanisms.

24. Learning • Chunking • Is the most developed learning mechanism. • Is deductive and compositional. • Resolving impasses can lead to learn new rules (called chunks). • Reinforcement Learning • Knowledge source is feedback from the environment (reward). • Learn rules that generate preferences based on future expected rewards. • Two parts in Soar • Must learn rules that test the appropriate features of the states and operators. • Initially create rules based on the rules that propose operators, and specialize them to consistently predict the same value. • Must learn the appropriate expected rewards for each rule. • Done by comparing the prediction of a rule with what happens during the next decision.

25. Learning • Episodic Memory • Knowledge source is the stream of experience. • Episodes are recorded automatically as a problem is solved. • An episode consists of a subset of the WM elements that exist at the time of recording. • Comparison with chunking and reinforcement learning • Passive learning mechanism. • The contents of an episode are determined only indirectly by reasoning. • No distinction between conditions for retrieval and what should be retrieved. • Semantic Memory • Knowledge source is the co-occurrence of structures in WM. • Knowledge about the rules of baseball, what is a home run, etc. • When to store a structure in semantic memory is a research issue. • Comparison with chunking and reinforcement learning • Deals with static structure instead of derivation-based rules. • Comparison with episodic memory • More general than episodes. • Place and time information is disassociated.

26. Putting it all together: a Soar Model of Joe Rookie • To Build a Full Model. • Specify the domain knowledge and which memories it is stored in. • Tie the domain knowledge to state structures and operators. • Specify the relationships between different levels by the impasses and the kinds of knowledge that will be missing, and learned.

27. Stepping back: the Soar Architecture in review • A Cognitive Architecture • Is a fixed set of mechanisms and structures that process content to produce behavior. • Is a theory, or point of view, about what cognitive behaviors have in common. • Soar Architecture • States and Operators • Working Memory • Long-term memory • The Perception/Motor Interface • The Decision Cycle • Impasses • Four Learning Mechanism

28. From Architecture to Unified Theories of Cognition • The model of Joe Rookie is a content theory. • Can explain why human needs change with time. • Can explain the specific factors that motivate people. • Soar has contributed a number of content theories to the field. • NTD-Soar, Instructo-Soar, IMPROV, TacAir-Soar, Soar MOUTBOT, etc. • Content theories model aspects of human language (or concept learning, or multi-tasking) within a framework. • The theory of the resulting model will be compatible with what is assumed to be architectural in the other content theories. • Content theories constitute a burgeoning unified theory of cognition (UTC). • Soar as a UTC.

29. Discussion • What is the similarity and difference between Soar and ACT-R? • Both try to model human. • Soar has focused on memory system, but ACT-R on brain image. • Do you think Soar can be a UTC? • A UTC must explain the following things. • How intelligent organisms flexibly react to stimuli from the environment. • How they exhibit goal-directed behavior and acquire goals rationally. • How they represent knowledge (or which symbols they use). • Learning. • It is known that there is efforts to model emotions into Soar. Do you think emotions can play a big role in cognition?