Multi-Agent Systemen Architecturen

1. Multi-Agent Systemen Architecturen dr. ir. M. Maris, Multi-agent systems

2. Reactive versus Hybrid agents • Reactive: no plans, thinking, memory. Only reaction; • Hybrid: There are reactive and other types of control layers. Other layers may be planning, reasoning, belief, learning. dr. ir. M. Maris, Multi-agent systems

3. Typical reactive agent control structure:Subsumption architecture • Use of “Behaviors” • Vertical modules • Interactions are fixed and defined by priorities of the behavioral modules • The modules can inhibit other modules Behavior x (Move to light) - Behavior 2 (Avoid obstacles) Output: Action to be executed - Behavior 1 (Grap cola can) Brooks, 1986 dr. ir. M. Maris, Multi-agent systems

4. Other reactive control architectures • Mars explorer (Steels) • PENGI (Agre and Chapman) • Situated Automata (Rosenschein and Kaelbling) • Ant-based dr. ir. M. Maris, Multi-agent systems

5. Limitations of reactive agents • There are no internal models, all necessary information must be extracted by the agents from their environment; • Agents can only take a short-term view (no-planning, belief); • They cannot learn from experience; • Intelligence must emerge from the interaction with the environment and that is hard to engineer and hard to understand. Neither is there any design methodology, only trial and error (examples: ants, cleaning robots); • The subsumption architecture is difficult to use if there are many (reactive) behaviors dr. ir. M. Maris, Multi-agent systems

6. Hybrid agent-architecture Nth intelligent layer 1st intelligent layer Reactive layer dr. ir. M. Maris, Multi-agent systems

7. Hybrid agents: Typical internal agent modules or layers • Perception / motor • Communication • Basic model of environment and other agents • Knowledge and experience, believes • Planning • Task performing • Reactive dr. ir. M. Maris, Multi-agent systems

8. Hybrid agents: Layers of control dr. ir. M. Maris, Multi-agent systems

9. Example hybrid architecture:Reactive with horizontal layering Agent Internal representations Decision making Planning Perception Action(s) Reactive dr. ir. M. Maris, Multi-agent systems

10. Example2, horizontal layering:Touring Machines (by Innes Ferguson) dr. ir. M. Maris, Multi-agent systems

11. Example two-pass control vertical layering:InteRRap (by Jörg Müller) dr. ir. M. Maris, Multi-agent systems

12. Blackboard architecture The architecture is based on individual modules that communicate not directly but by using a “blackboard” KS1 Environment Black- board KS2 KS3 control • There are three main subsystems: • Knowledge Sources (KSs) • The shared blackboard, used by the Knowledge Sources • Control device for managing conflicts of access (compare with Wiki) dr. ir. M. Maris, Multi-agent systems

13. Competitive tasks Task selector Task x (Move to light) Sensors Task 2 (Avoid obstacles) Actuators Task 1 (Grap cola can) Task switch dr. ir. M. Maris, Multi-agent systems

14. Production systems If <list of conditions> then <list of actions> agent Rule base Inference engine Execution Perception Database Environment dr. ir. M. Maris, Multi-agent systems

15. Connectionist architectures • Based on the metaphor of the brain • Using “neurons” as nodes (neural networks) • Are adaptive by changing the “weight” between two neurons • Many types of networks dr. ir. M. Maris, Multi-agent systems

16. Connectionist architectures (2) Example of a three-layer network Out In Internal (hidden) layer output layer input layer dr. ir. M. Maris, Multi-agent systems

17. Dynamic systems architectures The new action depends on the old action New action = old action + x Or: Action(t+1) = Action(t) + x This method tries to model the underlying agent system as accurate as possible, including real-time aspects dr. ir. M. Maris, Multi-agent systems

18. Multi-agent Systems Utility dr. ir. M. Maris, Multi-agent systems

19. Typical multi-agent system dr. ir. M. Maris, Multi-agent systems

20. The idea of Utility Utility is at the individual agent level and serves the goals of that agent dr. ir. M. Maris, Multi-agent systems

21. Preference and Utility Let’s name utility for agent i : ui Let there be a number of states w that an agent can be in. And W is the set of possible states: W = {w1 , w2 , …..} Then: ui(w) is the utility for the agent i in state w Usually, some states are preferred above other states, or: ui(w1) > ui(w2) (or, short form, w1 > w2 ) dr. ir. M. Maris, Multi-agent systems

22. Preference and Utility • Reflexivity: for all wW it holds that w1iw2 • Transitivity: if wiw2 and w2i3 than w1i3 • Comparability: for all wW it holds that either w1iw2 or w2 iw1 So the w’s in W can be put into a certain order of preference, with properties: dr. ir. M. Maris, Multi-agent systems

23. Utility in multi-agent systems Suppose that each agent can either do action D (defect) or C (cooperate): Ac = {C,D} Now define a transformation function t that maps the actions of the participating agents on a set of environmental states W: t: Ac (agent i) x Ac(agent j)  W dr. ir. M. Maris, Multi-agent systems

24. Example of utility in a MAS Env 1: t(D,D) = w1; t(D,C) = w2; t(C,D) = w3; t(C,C) = w4 Env 2: t(D,D) = w1; t(D,C) = w1; t(C,D) = w1; t(C,C) = w1 Env 3: t(D,D) = w1; t(D,C) = w2; t(C,D) = w1; t(C,C) = w2 For different environment, the actions C and D may result in differences in outcome (2 agents): dr. ir. M. Maris, Multi-agent systems

25. Numeric example of utility in MAS Suppose that: ui(w1) = 1, ui(w2) = 1, ui(w3) = 4, ui(w4) = 4 uj(w1) = 1, uj(w2) = 4, uj(w3) = 1, uj(w4) = 4 We now write: w1=t(D,D) = (D,D); w2= (D,C); w3= (C,D); w4= (C,C) and so: dr. ir. M. Maris, Multi-agent systems

26. Numeric example of utility in MAS ui(D, D) = 1; ui(D, C) = 1; ui(C, D) = 4; ui(C, C) = 4 uj(D, D) = 1; uj(D, C) = 4; uj(C, D) = 1; uj(C, C) = 4 It follows that agent’s i preference is : C,D i C,C i D,D i D,C and agent’s j: D,C j C,C j D,D j C,D dr. ir. M. Maris, Multi-agent systems

27. Utility in a Payoff Matrix i defects i cooperates 1 4 j defects 1 1 1 4 j cooperates 4 4 top-right is payoff received by agent i bottom-left is payoff received by agent j dr. ir. M. Maris, Multi-agent systems

28. Strategies Dominance: Create two subsets W1 and W2 out of W W1 dominates W2 if every outcome in W1 is preferred by the agent over every outcome in W2, or: 1  1 and 2  2 it holds that w1 > w2 dr. ir. M. Maris, Multi-agent systems

29. Actions and strategies • We define actions (set of possible actions Ac) as strategies si • We can say that s1 dominates s2 if the outcomes from s1 are preferred by the agent over the outcomes of s2 • Often one strategy will dominate the other • Between agents, a Nash equilibrium can exist dr. ir. M. Maris, Multi-agent systems

30. Nash equilibrium Two strategies s1 and s2 are in a Nash equilibrium if • Under the assumption that agent i plays s1, agent j can do no better than s2; and, • Under the assumption that agent j plays s2 agent i can do no better than to play s1 dr. ir. M. Maris, Multi-agent systems

31. Competitive / zero-sum interactions ui(w) + uj(w) = 0 for    Example: Chess W = {wi, w2} = {win, loose} Zero-sum interactions are always competitive dr. ir. M. Maris, Multi-agent systems

32. Prisoner’s dilemma Two man in prison (they cannot communicate with each other) are offered: • If you confess to the crime and the other doesn’t, you are free and the other will stay in jail for 3 years; • If you both confess, you both will get 2 years; • If neither confesses, both will get 1 year dr. ir. M. Maris, Multi-agent systems

33. Payoff Matrix Prisoner Dilemma i confess (defects) i not_confess (cooperates) j confess (defects) 2 3 0 2 j not_confess (cooperates) 0 1 3 1 Utility in number of years (boek rekent dat om, p115) What is the best strategy? Nash equilibrium! dr. ir. M. Maris, Multi-agent systems

34. Other symmetric game: Stag Hunt Your friend and you want to appear on the last day of school with green hair, as a joke. i defects i cooperates 1 0 j defects 2 1 2 3 j cooperates 0 3 dr. ir. M. Maris, Multi-agent systems

35. Dependence relations in MAS • Independent. They don’t depend on each other (or: their utilities do not depend on each other); • Unilateral. One agent depends on the other, but not vice versa; • Mutual. They depend on each other regarding a shared goal; • Reciprocaldependence. They depend on each other regarding a certain goal. dr. ir. M. Maris, Multi-agent systems

36. Summarizing • Utility gives a good measure of agent’s performance; • Utility helps the agent to order the preferences of its states; • Utility supports Multi-agent systems to reach the shared goal(s) dr. ir. M. Maris, Multi-agent systems

37. Volgende week Bayesiaanse netwerken dr. ir. M. Maris, Multi-agent systems