Metacognition for Effective Deliberation in Artificial Agents

1. Metacognition for Effective Deliberation in Artificial Agents Darsana Josyula 18 November 2011 Bowie State University

2. Artificial Agents • Deliberation • Process by which agents plan the tasks to perform in order to accomplish current goals • Action • Process by which agents perform each task in the plan

3. Deliberation Time • Static Environments • Deadlines • Dynamic Environments • Change in Environment • Preconditions not in effect • Plans not effective

4. Approaches to Managing Deliberation Time • Hybrid architectures with deliberative, reactive and action selection components • Gerhard Lakemeyer (Golog for Robotic Soccer) • Deliberation time decided by the action selection component • Anytime algorithms (estimates the efficiency of a solution as a factor of algorithm run time) • Nikos Vlassis • Deliberation time decided by the process that invokes the anytime algorithm

5. Approaches to Deliberation Time Management • Metacognition • Awareness of one's own thoughts and the factors that influence one's thinking • Based on estimates of allowable deliberation and action times • Monitoring deliberation and action • Making adjustments to deliberation and action processes

6. Metacognition in Deliberation and Action Metacognition Deliberation Action

7. Factors that influence Metacognition • Goals • Emotions • Resource Constraints • Plans • Performance Optimization • Influence of other Agents

8. Goals • Set of goals to be achieved • Choosing relevant goals • Type of goals • Mandatory versus Desirable (Needs versus Wants) • Desires versus Intentions (BDI agents) • Desires -> Intentions -> Actions • Maintenance versus Achievement • Conflicting goals • Source of goals • Progress towards goals

9. Metacognitive Monitoring and Control of Goal Processing • Marking the type of goals • Setting priorities for goals • Maintaining Expectations on progress towards goal

10. Metacognitive Monitoring and Control of Goal Processing • Expectation failures • Possible anomalies to be evaluated • Create new goals to deal with expectation failures • NAG cycle • MCL • Indications, Failures and Resonses

11. Emotions • Animals that exhibit more emotional behavior tend to be better suited for survival • D. Keltner, “Darwin’s touch: Survival of the kindest,” Psychology Today, February 11 2009. • Emotion has been shown to alter the cognitive process in human beings allowing for different responses to the same problem based on different emotional states. • L. Berkowitz, Causes and Consequences of Feelings,Cambridge University Press, 2000.

12. Emotions - Russel's Circumplex model of Positive and Negative Affect • Affective states organized in a circular structure in a 2D plane • Emotions arise from cognitive interpretations of neural sensations that are the product of two independent neurophysiological systems • These neurophysiological systems correspond to the pleasure axis and activation axis in the circumplex model

13. Emotions - Russel's Circumplex model of Positive and Negative Affect • The circumplex model of affects is consistent with findings in cognitive neuroscience, neuroimaging, and developmental studies of affects. • The circumplex model has been used to study the development of affective disorders as well as the genetic and cognitive underpinnings of affective processing within the central nervous system.

14. Emotions– Pleasure Axis Transitions • Represent the agent’s feelings about its own performance • Correspond to the number of expectation violations that occur • When no expectation violation occurs, the agent is pleased with its performance and hence moves its state to the right on the pleasure axis • When expectation violations occur, the system is frustrated by its inability to quell the violations and hence moves to the left on the pleasure axis • The intensity of the expectation violation (the difference between the observed value and the expected value) decides how far to the left the system moves with respect to its current emotional state

15. Emotions - Activation Axis Transitions • The activation axis transitions are based on the observations of the system and represent the system’s feeling of stress • As the number of observables that the system has to deal with increases, the system becomes more stressed and hence its emotional state moves upward in the activation axis • As the number of observables decrease, the system can relax and hence its emotional state moves downward in the activation axis

16. Metacognitive Monitoring and Control of Emotions • Monitoring number of observables • Maintaining Expectations on number of observables • Monitoring number of expectation violations

17. Metacognitive Monitoring and Control of Emotions • In the MCL model, failure nodes activate a set of possible responses and instantiates the highest utility response that corresponds to the type of failure. • Which action is deemed to be the best is a learned metric that could be altered for different emotional states; for instance, when stressed the best action may simply be the quickest, when relaxed the best action may be the slowest.

18. Plans/Actions to Perform • How a goal is achieved • Is a plan to achieve the goal known ? • If a plan is unknown, agent has to create a plan • Are the pros and cons of the plans known or unknown? • Which plan is better? • Success rate • Costs • Resource Usage • Conflicting plans

19. Metacognitive Monitoring and Control of Plans • Monitoring the success rate of plans adopted • Maintaining expectations on rate of success for plans adopted • Monitoring the actual costs for adopted plans and resource usage • Maintaining plan cost expectations • Monitoring the resource usage of adopted plans • Maintaining resource usage expectations • Monitoring Contradictions • Active Logic

20. Resource Constraints • Time Constraints • Deadlines • Resource Constraints • Minimize resource usage

21. Metacognitive Monitoring and Control of Resource Constraints • Monitoring passage of Time • Maintaining expectations on time requirements • Active Logic • Monitoring usage of Resources • Minimize resource usage

22. Influence of Other Agents • Competitive versus Cooperative • Cooperative agent changing to competitive or vice versa

23. Metacognitive Monitoring and Control of Influence of other Agents • Maintaining Expectations on the influence of other agents

24. Performance Optimization • May adversely influence resource constraints • Very important in competitive settings, but may be important in single agent settings as well • Examples • Get better in achieving a goal • Win against an opponent • Collect more rewards even at the expense of spending more resources

25. Metacognitive Monitoring and Control of Performance Optimization • Monitoring Performance Metrics • Maintaining Expectations of Performance Metrics

26. Metacognition – Vedantic Underpinnings • Goals • (Kāma – desire / goal to be achieved) • Emotions • (Krōdha – anger / emotional state) • Resource Constraints • (Lōbha – greed / miserliness) • Plans • (Mōha – delusion / Not seeing pros and cons) • Performance Optimization • (Mada – pride / vanity) • Influence of Other Agents • (Mātsarya – competitiveness)

27. Conclusion Monitoring Expectations is the key?