1 / 44

Partial-information Games with Reachability Objectives

Partial-information Games with Reachability Objectives. Krishnendu Chatterjee Formal Methods for Robotics and Automation July 15, 2011 . TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A. Games on Graphs.

damali
Download Presentation

Partial-information Games with Reachability Objectives

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Partial-information Games with Reachability Objectives KrishnenduChatterjee Formal Methods for Robotics and Automation July 15, 2011 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA

  2. Games on Graphs • Games on graphs. • Perfect-information graph games • Zermelo’s theorem about Chess in 1913 • From every configuration • Either player 1 can enforce a win. • Or player 2 can enforce a win. • Or both players can enforce a draw.

  3. Chess: Games on Graph • Chess is a game on graph. • Configuration graph.

  4. Graphs vs. Games Two interacting players in games: Player 1 (Box) vs Player 2 (Diamond).

  5. Game Graph

  6. Game Graphs • A game graph G= ((S,E), (S1, S2)) • Player 1 states (or vertices) S1 and similarly player 2 states S2, and (S1, S2) partitions S. • E is the set of edges. • E(s) out-going edges from s, and assume E(s) non-empty for all s. • Game played by moving tokens: when player 1 state, then player 1 chooses the out-going edge, and if player 2 state, player 2 chooses the outgoing edge.

  7. Game Example

  8. Game Example

  9. Game Example

  10. Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1 D(S). • ¼: S*S2! D(S).

  11. Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1! D(S). • History dependent and randomized. • History independent: depends only current state (memoryless or positional). • ¾: S1! D(S) • Deterministic: no randomization (pure strategies). • ¾: S*S1! S • Deterministic and memoryless: no memory and no randomization (pure and memoryless and is the simplest class). • ¾: S1! S • Same notations for player 2 strategies ¼.

  12. Objectives • Objectives are subsets of infinite paths, i.e., õ S!. • Reachability: there is a set of good vertices (example check-mate) and goal is to reach them. Formally, for a set T if vertices or states, the objective is the set of paths that visit the target T at least once.

  13. Applications: Verification and Control of Systems • Verification and control of systems • Environment • Controller M satisfies property (Ã) E C

  14. Applications: Verification and Control of Systems • Verification and control of systems • Question: does there exists a controller that against all environment ensures the property. C M E satisfies property (Ã) || ||

  15. Applications: Systems for Specification • Synthesis of systems from specification • Input/Output signals. • Automata over I/O that specifies the desired set of behaviors. • Can the input player present input such that no matter how the output player plays the generated sequence of I/O signals is accepted by automata ? • Deterministic automata: Games.

  16. Game Models Applications -synthesis [Church, Ramadge/Wonham, Pnueli/Rosner] -model checking of open systems -receptiveness [Dill, Abadi/Lamport] -semantics of interaction [Abramsky] -non-emptiness of tree automata [Rabin, Gurevich/ Harrington] -behavioral type systems and interface automata[deAlfaro/ Henzinger] -model-based testing [Gurevich/Veanes et al.] -etc.

  17. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

  18. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

  19. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

  20. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. • Fix-point X T

  21. Reachability Games • Winning set for a partition: Determinacy • Player 1 wins: then no matter what player 2 does, certainly reach the target. • Player 2 wins: then no matter what player 1 does, the target is never reached. • Memoryless winning strategies. • Can be computed in linear time [Beeri 81, Immerman 81].

  22. Chess Theorem • Zermelo’s Theorem Win1 Win2 Both draw

  23. Partial-information Graph Games

  24. Why Partial-information • Perfect-information: controller knows everything about the system. • This is often unrealistic in the design of reactive systems because • systems have internal state not visible to controller (private variables) • noisy sensors entail uncertainties on the state of the game • Partial-observation Hidden variables = imperfect information. Sensor uncertainty = imperfect information.

  25. Partial-information Games • A PIG G =(L, A, , O) is as follows • L is a finite set of locations (or states). • A is a finite set of input letters (or actions). • µ L £ A £ L non-deterministic transition relation that for a state and an action gives the possible next states. • O is the set of observations and is a partition of the state space. The observation represents what is observable. • Perfect-information: O={{l} | l 2 L}.

  26. PIG: Example b a a,b b a

  27. New Solution Concepts • Sure winning: winning with certainty (in perfect information setting determinacy). • Almost-sure winning: win with probability 1. • Limit-sure winning: win with probability arbitrary close to 1. • We will illustrate the solution concepts with card games.

  28. Card Game 1 • Step 1: Player 2 selects a card from the deck of 52 cards and moves it from the deck (player 1 does not know the card). • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret card or goes back to Step 2 a. • Player 1 wins if the guess is correct.

  29. Card Game 1 • Player 1 can win with probability 1: goes back to Step 2 a until all 51 cards are seen, and plays uniformly at random for choosing cards till then. • Player 1 cannot win with certainty: there are cases (though with probability 0) such that all cards are not seen. Then player 1 either never makes a guess or makes a wrong guess with positive probability.

  30. Card Game 2 • Step 1: Player 2 selects a new card from an exactly same deck and puts is in the deck of 52 cards (player 1 does not know the new card). So the deck has 53 cards with one duplicate. • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret duplicate card or goes back to Step 2 a. • Player 1 wins if the guess is correct.

  31. Card Game 2 • Player 1 can win with probability arbitrary close to 1: goes back to Step 2 a for a long time and then choose the card with highest frequency (choosing uniformly at random till the choice is made). • Player 1 cannot win probability 1, there is a tiny chance that not the duplicate card has the highest frequency, but can win with probability arbitrary close to 1, (i.e., for all ² >0, player 1 can win with probability 1- ², in other words the limit is 1).

  32. Sure winning for Reachability • Result from [Reif 79] • Memory is required. • Exponential memory required. • Subset construction: what subsets of states (knowledge) player 1 can be. Reduction to exponential size perfect-information games. • EXPTIME-complete.

  33. Partial-information Games a b a • In starting play a. • In yellow play a and b at random. • In purple: • if last was yellow then a • if last was starting, then b. • Requires both randomization and memory a b a b a a b a b

  34. Almost-sure winning for Reachability • Result from [CDHR 06, CHDR 07] • Standard subset construction fails: as it captures only sure winning, and not same as almost-sure winning. • More involved subset construction is required. • EXPTIME-complete.

  35. Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • Given a knowledge cannot play an action that is unsafe (takes out of the winning knowledge set). • Must play safe actions. • Play all safe actions uniformly at random.

  36. Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • What GOD can do we can do by non-determinism. • This gives NEXPTIME algorithm, for EXPTIME algorithm see [CDHR 07].

  37. Summary: Theory of Graph Games

  38. Limit-sure winning for Reachability • Limit-sure winning for reachability is undecidable [GO 10, CH 10]. • Reduction from the Post-correspondence problem (PCP).

  39. Limit-sure winning for Reachability • The undecidable result is first proved for probabilistic automata. • Probabilistic automata are special case of blind stochastic games. • For partial-information games, stochastic transitions can be simulated by deterministic transition [CDGH 10]. • Consequently we obtain the undecidability result.

  40. Summary: Theory of Graph Games

  41. Conclusion and Related Problems • Theory of graph games • Perfect-information and partial-information games. • Different solution concepts and different complexity. • Several algorithmic questions open. • Partial-observation stochastic games with pure strategies (Working manuscript of CD 11) • Many surprising results (such as non-elementary complete memory bounds).

  42. Conclusion and Related Problems • Partial information games • Problem with clear practical motivation. • Challenging to establish the right frontier of complexity. • Important generalization of perfect-information games. • Unfortunately, undecidable and also high complexity. • Looking for new definitions (of sub-classes).

  43. Collaborators • Laurent Doyen • Hugo Gimbert • Thomas A. Henzinger • Jean-Francois Raskin

  44. The end Thank you ! Questions ?

More Related