A Personal Odyssey in the World of Multi-Agent Research

1. A Personal Odyssey in the World of Multi-Agent Research Victor R. Lesser Computer Science Department University of Massachusetts, Amherst July 26, 1999

2. Theme • To provide a personal perspective on how a research career develops • How ideas get created and evolve over time • My personal research agenda • Thoughts on where the field should be going

3. Creation and Evolution of MAS/DAI Approaches • Functionally/Accurate Cooperative (FA/C) paradigm (Corkill and Carver) • Tolerance of inconsistency • Range of acceptable answers • Error resolution through exchange of partial and tentative subproblem solutions • Layered Agent Control (Corkill) • Organizational structuring • Satisficing agent coordination • Interplay between local and non-local control

4. MAS/DAI ideas (continued) • Partial Global Planning (PGP) and its successor GPGP as a framework for real-time agent coordination (Durfee & Decker) • Quantitative view of coordination based on a distributed search model • Taxonomy of Subproblem interactions • Agent Task Language —TAEMS • Fault-detection, diagnosis and adaptation as mechanism for coordination adaptation (Hudlicka & Sugawara) • Negotiation as a distributed search • Multi-stage negotiation (Conry, Kuwabara & Meyer) • Level-commitment as protocol for search by self-interested agents (Sandholm)

5. Precursor Research • Multi-computer operating system (1963-65) • Excitement and complexity • Reconfigurable Multi-processor Architecture (1966-1972) • Dynamic Mapping of Process network on to Processor network • Hardware kernel micro-operating system • Mapping had to take into account of interrelationship among processes • Issues of scale • Control working set • Importance of careful attention to empirical data • Detail simulation

6. Hearsay-II: Parallel, Cooperating Knowledge-Source Model (1973-1975) • Blackboard Architecture

7. Functional Descriptionof the Speech-Understanding KSs

8. Key Ideas in Blackboard Architecture • Distributed, multi-level asynchronous search • Integrated search at different abstraction levels • Error-resolution through search and combining of approximate knowledge • Sophisticated control reasoning • Use of approximate knowledge for control • Probabilistic view of search control Ideas and lesson learned in HS-II will be an important influence on future work

9. Transition to Distributed AI research — the importance of serendipity • Parallel Processing Experiments (Fennel) • The Wisdom of Allen Newell in forbidding me to talk • Conversation with Bob Kahn (one of the founders of the Internet)

10. Distributed Hearsay-II Experiments (Erman, 1977-1979) • Three node System with Overlapping Acoustic Data • Communication of only High-Level Results • No changes to basic architecture except for transmit and receive KSs

11. sensor1 sensor2 sensor3 Network of Hearsay-II Systems

12. Results of Experiments • Reproduced Results of Centralized System • Slight Speed up and Reduced Communication • Robustness in face of errorful communication channel • Handled 35% error rate

13. Lessons — It worked but...! • Lack of coherent behavior • Distraction • Inappropriate Communication/Computation • Redundant • Lack of timeliness • Lack of Focus • Simplistic Local and Network Control Inadequate • Local agent control needs to be more sophisticated when taking into account interactions with other agents

14. Key Question that Focused Research in the 1980’s • Can computationally tractable cooperative strategies be developed that maintain both coherent agent activity and system robustness? Implicit recognition of tension between reactive and reflective control

15. DVMT: an Environment for Research in Cooperative Distributed Problem Solving • Build and evaluate more complex local agent control, coordination strategies and organization strategies

16. DVMT: Agent Architecture(Corkill, 1983) • Static Meta-level Control • Organizational structuring • Goal-Directed requests for information • Integrating external and internal requests for processing

17. Integrating Data and Goal-Directed Control and Organizational Structuring

18. Limitations of Static Meta-Level control (1987) • Transmission of meta-level state information — only partially successful • Missing information about future activities • Transmission of activity plans • Partial global planning (Durfee)

19. PartialGlobalPlanningArchitecture

20. Partial Global Planning(Durfee) • Representation of near-term agent activities • Intermediate goals of activities • Region and likely vehicles • Behavioral characteristics • Timing and likelihood of success • Relationship of interagent activities • Spatial overlapping and adjacent interpretation regions • Basis for reorganizing local activities • Exploiting predictive information • Avoiding redundant activities • Allow for load sharing

21. Some Important Digressions in Local Agent Control • Meta-level control through Diagnosing of Problem-Solving Behavior (Hudlicka, 1984) • Led to work on learning new situation specific coordination rules via detection and diagnosis (Sugawara, 1993) • RESUN — a framework for problem solving control based on symbolic reasoning about source of uncertainty (Carver, 1989) • Led to work on DRESUN which provide a distributed framework for focused communication to resolve inconsistent agent beliefs (Carver, 1992-1997)

22. Further Work on Local Control • Design-to-Time Scheduling (Garvey, Wagner) • Approach to real-time agent control by dynamically constructing a schedule of activities to meet real-time deadlines • Exploit the existence of alternative algorithms that trade off quality of solution for resource usage • Led to deeper understanding of the issues of uncertainty Techniques for Sophisticated Local Control have strong implications for Non-Local Control

23. Digressions into the World of Negotiation Trying to understand cooperative and self-interested contracting • Multi-stage negotiation (Conry et al., Lander, Laasris, Moehlman, Neiman) • Cooperative dialogue among agents (1987-1997) • Self-Interested Agent Interaction (Sandholm) • Level commitment negotiation protocol Digressions are sometimes important for validating old intuitions and gaining new ones

24. Key Questions that Focused Research in the 1990’s • Is there some deeper theory of Agent Coordination implicit in this work • Can we create infrastructure/ frameworks that eliminate a lot of the work in constructing MAS systems The search for the Holy Grail!

25. Agent1 Agent2 G10 G20 G11 G12 . . . . . . G1k G1,2m G2p . . . . . . . . . . . G2t G11,1 G11,2 G1m,1 G2m,2 G2p,1 G2p,2 G1m,1,1 G1m,1,2 G2p,1,3 (G2p,1,4) G2p,2,2 Generalizing PGP(Decker) OR AND AND AND OR AND OR AND d2j+1 …………………………………………….d2z d11 ………………………………………. d1j DATA/ Resources A distributed goal search tree involving Agent1 and Agent2. The dotted arrows indicate interdependencies between goals and data in different agents, solid arrows dependencies within an agent. The superscripts associated with goals and data indicate the agent which contains them (Jennings, 1993).

26. TAEMS: A Domain Independent Framework for Modeling User Activities • The top-level goals/objectives/abstract-tasks that an agent intends to achieve • One or more of the possible ways that they could be achieved, expressed as an abstraction hierarchy whose leaves are basic action instantiations, called methods • A precise, quantitative definition of the degree of achievement in terms of measurable characteristics such as solution quality and time

27. TAEMS (continued) • Task relationships that indicate how basic actions or abstract task achievement affect task characteristics (e.g., quality and time) elsewhere in the task structure • Hard relationships (e.g., enables) denote when the result from one problem-solving activity is required to perform another, or when performing one activity precludes the performance of another • Soft relationships (e.g. facilitates) express the notion that the results of one activity may be beneficial (or harmful) to another activity, but that the results are not required in order to perform the activity. • The resource consumption characteristics of tasks and how a lack of resources affects them.

28. Information Gathering Example

30. Important Aspects of TAEMS • Abstract View of Agent Activities • level of detail necessary for understanding interactions and scheduling decisions • Relationships among activities based on data flow: enables, facilitates, favor, disable, etc. • Relationships among activities how they contribute to the overall goal — quality accumulation functions: min, max, sum, etc. • Schedule represents policy — guidance for resource consumption and goals • Worth Oriented • Coordination as an Optimization Problem • Real-time deadlines

31. WARREN Style Model of Multi-Agent Information Gathering

32. GPGP Agent Architecture

33. Clear Separation of Local Control from Coordination • Coordination is generation of commitments • Importance, utility, negotiability, decommitment • Commitments lead to constraints on local scheduling • Earliest start time of a task • Deadline for completion of a task • Interval when can’t be executed

34. No person is an island unto themselves! Integrating GPGP with other Approaches

35. Putting it all together — An architecture for Large Agent Societies

36. Personal Perspective on MAS based on this research path • Agent Flexibility in Open Environments • Agents need to be able to adapt their local problem solving to the available resources and goals of the system. • Long-term learning needs to be an integral part of an agent architecture • Agents not restricted to solving one goal at a time but may flexibly interleave their activities to solve multiple goals concurrently • Error resolution/management needs to be integral part of agent problem solving • Satisficing control • Less than optimal but still acceptable levels of coordination among agents is traded off for a significant reduction in computational costs to implement cooperative control. • Emphasis on satisficing behavior subtly moves the focus from the performance of individual agents to the properties and character of the aggregate behavior of agents.

37. Personal Perspective (continued) • Predicting Performance of MAS systems is possible via probabilistic analysis • Requires detail model of the environment • Interaction between local and non-local agent control • For effective agent coordination local agent control must have a certain level of sophistication in order to be able to understand what it has done, what is currently doing and what it intends to do • Agent Roles and Responsibilities for large agent societies • organizing agents in terms of roles and responsibilities can significantly decrease the computational burden of coordinating their activities.

38. Personal Perspective (continued) • Centrality of Commitment to coordinated behavior • Both long- and short-term coordination can be viewed in terms of commitments that have varying duration and specificity. • Layered Control • Modulation—higher layers providing constraints (policies) to lower levels that modulate (circumscribe) their control decisions • Bi-directional Interaction(negotiation) among Layers — Though constraints flow down the layers, information that flows in the other direction allows these constraints to be modified in case they can’t be met or they lead to inappropriate behavior

39. Personal Perspective (continued) • Situation-specificity • There is no one best approach to organizing and controlling computational activities for all situations when the computational and resource costs of this control reasoning is taken into account. • Quantitative View of Coordination • Efficient and effective coordination must account for the benefits and the costs of coordination in the current situation. • Coordination can be seen as a distributed mechanism for approximating a global optimization problem of task assignment

40. Personal Perspective (continued) • Domain-independence —The aspects of a domain that affect coordination can be abstracted and represented in a domain-independent language. • An agent’s goals and criteria for their successful performance • The performance characteristics and resource requirements of the alternative methods it possesses for accomplishing its goals, • Qualitative and quantitative interdependencies among its methods and those of other agents

41. Personal Perspective (continued) • Representing and Reasoning about Assumptions • To the degree that the system can either re-derive or explicitly represent the assumptions behind these control decisions • The more the system can effectively detect and diagnose the causes for inappropriate or unexpected agent behavior. • Importance of Experimentation — we are still an experimental science • Don’t yet have good ways to predict performance • Statistical analysis is important but don’t forget to look at the details

42. Current Research Projects • Organizational Structuring, Design and Adaptation for Large Agent Societies (Horling, Vincent, Wagner) • Real-time negotiation • Layered, Domain-Independent Coordination • GPGP-II (Wagner and Xuan) • JIL to GPGP (Raja and Zhang) • Distributed Situation Assessment/DRESUN • Satisficing termination (Carver) • Distributed Dynamic Bayesean Network and Influence Diagrams (Carver, Xiang, Zhang, Zilberstein)

43. Current Research Projects, Cont’d • Survivable MAS Systems (Xuan) • Coordinating for fault-tolerance • Distributed Markov Decision Processes • Resource Bounded Reasoning/Design-to-Criteria Scheduling (Raja and Wagner) • Application Focus: • Cooperative Information Gathering • Intelligent Home • Supply Chain Manufacturing

44. Questions on my mind • Is there a unified perspective with in which self-interest and cooperative agents can be understood? • Is that perspective distributed search? • How viable is it to think about emotions and power relationships as computational mechanisms making it possible to approximate the global optimal solution in a distributed way through local optimizations? • Early work on skeptical nodes • What type of meta-level framework (with limited and bounded computational overhead) will allow us reason about coordination costs as first-class objects so that it possible to dynamically balance problem-solving activities with coordination activities?

45. Questions on my mind (continued) • Are Markov Decision Processes a computationally viable approach for dynamic multi-agent coordination? • What knowledge and reasoning is necessary for designing top-down an agent organization? • In what situation can bottom-up evolutionary organizational structuring produce good organizations • Will the world of MAS/DAI be dominated in the future by game theoretic ideas and market mechanisms? • what is the role of cooperative agents?

46. MAS in the 21st Century—A Dominant Model Cooperating, Intelligent Agent Societies (seamless integration among people/machines) • Constructionist perspective • built out of heterogeneous, semi-autonomous agents • having varying motivations from totally self-interested to benevolent • High-level artificial language for cooperation • Problem solving for effective cooperation will be as or more sophisticated than the actual domain problem solving • reasoning about goals, plans, intentions, and knowledge of other agents

47. MAS in the 21st Century, Cont’d • Operate in a “satisficing” mode • Do the best they can within available resource constraints • Deal with uncertainty as an integral part of network problem solving • Complex organizational relationships among agents • Highly adaptive/highly reliable • Learning will be an important part of their structure (short-term/long-term) • Able to adapt their problem-solving structure to respond to changing task/environmental conditions Profound implications for AI & Computer Science!

48. Important Directions for the Field to Realize this Vision • Development of software infrastructure to help build sophisticated, interacting agents • What will it be wrappers, languages or frameworks or some combination? • Techniques for Sharing of knowledge/data among heterogeneous agents • Are ontologies the answer or will there be the need for more sophisticated knowledge translation approaches or specialized languages?

49. Directions (continued) • Mechanisms for dynamically establishing interaction protocols among heterogeneous agents • Are recent ideas such as “civil agent societies” by Dellacros and Klein a viable approach? • Analysis tool for understanding the performance of such systems before they are implemented • Design rules and mechanisms for agent societies so that they will not evolve in ways that lead to inappropriate behavior or poor performance

50. Parting Thoughts • This is a very exciting time for researchers in MAS • The practical application of this technology is here! • The set of ideas that the field has developed only scratch the surface • There is a tremendous amount of work to be done • There are a lot of hard problems to work on • Let your intuitions drive you — not what is necessarily currently in fashion