Compressing Mental Model Spaces and Modeling Human Strategic Intent

1. Compressing Mental Model Spaces and Modeling Human Strategic Intent

2. Prashant Doshi University of Georgia, USA

3. http://thinc.cs.uga.edu

4. Yifeng Zeng Reader, Teesside Univ. Previously: Assoc Prof., Aalborg Univ. Yingke Chen Doctoral student Muthu Chandrasekaran Doctoral student Hua Mao Doctoral student

5. What is a mental behavioral model?

6. How large is the behavioral model space?

7. How large is the behavioral model space? General definition A mapping from the agent’s history of observations to its actions

8. How large is the behavioral model space? 2H (Aj) Uncountably infinite

9. How large is the behavioral model space? Let’s assume computable models Countable A very large portion of the model space is not computable!

10. Daniel Dennett Philosopher and Cognitive Scientist Intentional stance Ascribe beliefs, preferences and intent to explain others’ actions (analogous to theory of mind - ToM)

11. Organize the mental models Intentional models Subintentional models

12. Organize the mental models Intentional models E.g., POMDP  =  bj, Aj, Tj, j, Oj, Rj, OCj BDI, ToM Subintentional models Frame (may give rise to recursive modeling)

13. Organize the mental models Intentional models E.g., POMDP  =  bj, Aj, Tj, j, Oj, Rj, OCj BDI, ToM Subintentional models E.g., (Aj), finite state controller, plan Frame

14. Finite model space grows as the interaction progresses

15. Growth in the model space Other agent may receive any one of |j| observations |Mj|  |Mj||j| |Mj||j|2  ...  |Mj||j|t 0 1 2 t

16. Growth in the model space Exponential

17. Absolute continuity condition (ACC)

18. ACC Subjective distribution over histories True distribution over histories

19. ACC is a sufficient and necessary condition for Bayesian update of belief over models

20. How do we satisfy ACC? Cautious beliefs (full prior) Grain of truth assumption Prior with a grain of truth is sufficient but not necessary

21. General model space is large and grows exponentially as the interaction progresses

22. It would be great if we can compress this space! • No loss in value to the modeler • Flexible loss in value for greater compression Lossless Lossy

23. Expansive usefulness of model space compression to many areas: • Sequential decision making (dt-planning) in multiagent settings • Bayesian plan recognition • Games of imperfect information

24. Interactive POMDP framework (Gmytrasiewicz&Doshi05) 1. Sequential decision making in multiagent settings Include models of the other agent in the state space Update beliefs over the physical state and models

25. General and domain-independent approach for compression Establish equivalence relations that partition the model space and retain representative models from each equivalence class

26. Approach #1: Behavioral equivalence (Rathanasabapathy et al.06,Pynadath&Marsella07) • Intentional models whose complete solutions • are identical are considered equivalent

27. Approach #1: Behavioral equivalence Behaviorally minimal set of models

28. Approach #1: Behavioral equivalence Lossless Works when intentional models have differing frames

29. Approach #1: Behavioral equivalence Impact on dt-planning in multiagent settings Multiagent tiger Multiagent MM Multiagent tiger

30. Approach #1: Behavioral equivalence Utilize model solutions (policy trees) for mitigating model growth Model reps that are not BE may become BE next step onwards • Preemptively identify such models and do not update all of them

31. Approach #2: -Behavioral equivalence (Zeng et al.11,12) Redefine BE

32. Approach #2: RevisitBE (Zeng et al.11,12) Intentional models whose partial depth-d solutions are identical and vectors of updated beliefs at the leaves of the partial trees are identical are considered equivalent Lossless if frames are identical Sufficient but not necessary

33. Approach #2: (,d)-Behavioral equivalence • Two models are (,d)-BE if their partial depth-d solutions are identical and vectors of updated beliefs at the leaves of the partial trees differ by  Models are (0.33,1)-BE Lossy

34. Approach #2: -Behavioral equivalence • Lemma (Boyen&Koller98): KL divergence betweentwo distributions in a discrete Markov stochastic process reduces or remains the same after a transition, with the mixing rate acting as a discount factor • Mixing raterepresents the minimal amount by which the posterior distributions agree with each other after one transition • Property of a problem and may be pre-computed

35. Approach #2: -Behavioral equivalence Given the mixing rate and a bound, , on the divergence between two belief vectors, lemma allows computing the depth, d, at which the bound is reached Compare two solutions up to depth d for equality

36. Approach #2: -Behavioral equivalence Impact on dt-planning in multiagent settings Discount factor F = 0.5 Multiagent Concert On a UAV reconnaissance problem in a 5x5 grid, allows the solution to scale to a 10 step look ahead in 20 minutes Multiagent Concert

37. Approach #2: -Behavioral equivalence What is the value of d when some problems exhibit F with a value of 0 or 1? F=1 implies that the KL divergence is 0 after one step: Set d = 1 • F=0 implies that the KL divergence does not reduce: Arbitrarily set d to the horizon

38. Approach #3: Action equivalence (Zeng et al.09,12) Intentional or subintentional models whose predictions at time step t (action distributions) are identical are considered equivalent at t

39. Approach #3: Action equivalence

40. Approach #3: Action equivalence Lossy Works when intentional models have differing frames

41. Approach #3: Action equivalence Impact on dt-planning in multiagent settings AE bounds the model space at each time step to the number of distinct actions Multiagent tiger

42. Approach #4: Influence equivalence (related to Witwicki&Durfee11) Intentional or subintentional models whose predictions at time step tinfluence the subject agent’s plan identically are considered equivalent at t Regardless of whether the other agent opened the left or right door, the tiger resets thereby affecting the agent’s plan identically

43. Approach #4: Influence equivalence Influence may be measured as the change in the subject agent’s belief due to the action Group more models at time step t compared to AE Lossy

44. Compression due to approximate equivalence may violate ACC Regain ACCby appending a covering model to the compressed set of representatives

45. Open questions

46. N > 2 agents Under what conditions could equivalent models belonging to different agents be grouped together into an equivalence class?

47. Can we avoid solving models by using heuristics for identifying approximately equivalent models?

48. Modeling Strategic Human Intent

49. Adam Goodie Professor of Psychology, UGA Yifeng Zeng Reader, Teesside Univ. Previously: Assoc Prof., Aalborg Univ. Yingke Chen Doctoral student Xia Qu Doctoral student Roi Ceren Doctoral student Muthu Chandrasekaran Doctoral student Matthew Meisel Doctoral student Hua Mao Doctoral student

50. Computational modeling of probability judgment in stochastic games Computational modeling of human recursive thinking in sequential games