Multiagent Systems

1. Multiagent Systems Game Theory

2. Non-Cooperative Game Theory • Game Theory is the study of the interaction and decision making of rational, self-interested agents • Each agent has its own interests • While these are mostly distinct, non-cooperative does not preclude that agents have common interests • Agents are rational and only pursue their interests • The individual agent is the basic unit • Collaborative or coalitional game theory model teams as basic units

3. Game Theory • TCP backoff example • Agents can use a correct or an incorrect TCP backoff mechanism • Correct mechanism backs off after package collisions • Incorrect mechanism does not increase backup over time • Collisions of packets from multiple agents can lead to delays due to retries • Correct implementation leads to an expected delay of 1 • Incorrect implementation immediately retries, causing a correct implementation to back off further, leading to a delay of 4 • Two incorrect implementations clash multiple times, leading to an expected delay of 3

4. Game Theory • The interaction and decision process of the agents is modeled as a game with separate utility functions for each agent • What action should each player take ? • Will both agents behave the same ? • Do the precise numbers influence behavior ? • Would communication change behavior ? • Should actions change when a game is played repeatedly ? • Is it important that agents are assumed to be rational ?

5. Single Shot Games • Single shot (or single stage) games are games where multiple agents have to make a single action choice. • Past experiences have no influence on the current game and the agents’ decisions • Utility is equal to the expected reward • Finite n-person game • N is a finite set of n agents (players) • is the joint action set of the agents • is the joint utility function for the agent

6. Matrix Games • A finite single stage game can be written in the form of an n-dimensional matrix (called “normal form”) • Each dimension enumerates the actions of one agent • Entries in the matrix are n-tuples of utility values for each agent for outcome of the given action choices • The TCP backoff example

7. Matrix Games • The prisoners’ dilemma

8. Canonical Games • Pure competition games are 2-player constant sum games • Every utility gain for one agent is an equal utility loss for the other agents • Only one utility function is needed • Example: Matching Pennies

9. Pure competition Games • Example: Rock, Paper, Scissors

10. Canonical Games • Coordination games are games where players have exactly the same interests • All players obtain the same utilities • Only one utility function is needed • Note: this is still non-cooperative since each agent makes its own decision • Example: Driving on a side of the road

11. Canonical Games • General games contain elements of cooperation and competition • Example: Battle of the sexes

12. Decision-Making: Analyzing Games • “Better” outcomes/strategies have to be defined • Utilities for different agents can not be compared • All agents are equally important • Pareto dominance • An outcome o Pareto-dominates an outcome o’ if for every agent outcome o is at least as good as outcome o’

13. Pareto Optimality • An outcome/strategy is Pareto optimal if it is not Pareto dominated by any other outcome • Every game has at least one Pareto optimal strategy • Every game has at least one Pareto optimal pure strategy • Games can have multiple Pareto optimal strategy

14. Pareto Optimality • Pareto Optimality allows to identify better strategies from an outside point of view. • Examples:

15. Decision-Making: Best Response • If an agent knows the strategies of all other agents it can easily pick its own action • Best response: • When the strategies of other agents are not known we have to look for “stable” strategies • A strategy is a (pure strategy) Nash equilibrium if for every agent its action is a best response to the actions of the other agents

16. Nash Equilibrium • A Nash Equilibrium is a stable strategy • Any agent deviating from it will not increase its utility • An agent that is aware of the other agents’ strategies can not increase its utility using this knowledge (the equilibrium is its best strategy) • In an equilibrium it is possible to determine the other agents’ strategies based on the assumption of rationality

17. Nash Equilibrium • Nash equilibria allow agents to identify stable strategies that lead to “best” outcomes. • Examples:

18. Nash Equilibrium • Many games do not have pure strategy equilibria • In competitive settings it is often a bad idea to always do the same • If other agents know your actions they can use this knowledge to utilize your weaknesses

19. Mixed Strategies • Agents can follow different strategies • Pure strategies are strategies where agents select actions deterministically • Mixed strategies are strategies where actions are taken according to a probability distribution • Strategy profiles are the joint strategies of all the agents

20. Utility of Mixed Strategies • The utility of a mixed strategy profile can be computed as the expected value of the outcome lottery

21. Best Response and Nash Equilibrium • If an agent knows the strategies of all other agents it can pick its own (mixed) strategy • Best response: • When the strategies of other agents are not known we have to look for “stable” strategies • A strategy profile is a Nash equilibrium if for every agent its strategy is a best response to the strategies of the other agents

22. Nash Equilibrium • A Strict Nash equilibrium is a Nash equilibrium where for each agent the strategy is a strictly better (higher utility) response to the other agents’ strategies than any other strategy • A Weak Nash equilibrium is any Nash equilibrium that is not a Strict Nash equilibrium • All mixed strategy equilibria are Weak

23. Nash Equilibrium • Nash (1951): Every finite game has at least one Nash equilibrium • Computing Nash equilibria can be complex • It becomes easier if the support of the strategy (i.e. the set of actions with probabilities greater than 0) is known

24. Solution Approaches • To address decision making in matrix games we need algorithms that can determine Nash equilibria • Maxmin and Minmax for constant sum games • Linear programming for (zero-sum) matrix games • Linear Complimentarity Problem (LPC) for general 2-player games • Lemke-Howson algorithm