Strategic Research Directions in AI: Distributed AI and Agent Systems

1. Strategic Research Directions in AI:Distributed AI and Agent Systems Edmund H. Durfee University of Michigan AI Laboratory

2. Agent Coordination and Control Should one or more agents notice? Should one or more agents respond?

3. Technical Challenge:Constrained Real-Time Responsiveness Cannot be prepared to notice and respond to everything at once. Goal: Maximize real-time responsiveness under operational and execution constraints. Example Technical Foundations: temporal constraint networks and MDPs. Example Strategic Direction: Constrained MDPs • Formulation constraints: Limited computational resources and/or time to formulate policies. • Operationalization constraints: Platform resources (perception, actuation, memory) limit size/complexity of policies. • Execution constraints: Consumable resources (power, fuel, bandwidth) limit duration/persistence of policies.

4. Constrained vs Unconstrained MDPs

5. Technical Challenge:Boundedly-Optimal MultiAgent Systems Responsiveness responsibilities can be distributed across multiple agents. Goal: Maximize system-wide real-time responsiveness under individual and collective constraints. Example Technical Foundations: single-agent bounded optimality techniques and MDPs. Example Strategic Direction: Multiagent MDPs co-designing for coordination protocols and individual decision-making • Growing activity on MAMDPs (UMass, USC, Toronto, UMich,…) • Example: Iterated convergence on compatible schedulable policies across agents.

6. acj ttac ttac aci I ttac ttac D acl ttac ack Convergence Protocol Example • Represent possible (re)actions of other agents as temporal transitions (ttac labels). • Both agents may handle dangerous state D.

7. acj ttac ttac aci I ttac ttac D acl ttac ack Convergence Protocol Example Knowing which reactions other agent plans can restrict which states this agent must worry about.

8. acj ttac ttac aci I ttac ttac D acl Convergence Protocol Example • Revelation of choices through protocol may eliminate entire subspace (and hence need to plan/schedule actions for those states).

9. acj ttac ttac aci I ttac ttac D Convergence Protocol Example • Revelation of choices through protocol may eliminate just a required action, by knowing other agent will handle the important event.

10. Technical Challenge:Social Autonomy Interdependence comes at a cost. Uncertainty over commitment fulfillment, and even definition! Goal: Automate striking an informed and flexible balance between risks and benefits of dependence. Example Technical Foundations: reflective architectures, adjustable autonomy, abstraction, commitment/convention. Example Strategic Direction: Modeling methods, protocols, and languages for coordination with strategic ignorance. • Revealing too many details incurs overhead. • Revealing too many details reduces local flexibility. • Revealing too few details encourages inefficiencies. • Revealing too few details increases risk.

11. Tradeoffs in Knowledge Revelation Each agent has a couple of routes from which to choose. A A B B

12. Tradeoffs in Knowledge Revelation Not revealing any information can be risky. A B

13. A B Tradeoffs in Knowledge Revelation Revealing specific plans could remove some kinds of risk, but could jeopardize success.

14. Tradeoffs in Knowledge Revelation Remaining vague about plans retains flexibility, but can reduce efficiency (less parallelism) A B

15. Technical Challenge:Relationship Discovery In larger, more emergent multiagent systems, relationships “happen” in often unexpected ways. Goal: Discover, represent, and resolve important relationships cost-effectively in sparsely interacting contexts. Example Technical Foundations: forwarding protocols, summarization, self-description languages, graphical games, multiagent learning, distributed CSPs. Example Strategic Direction: Emerging aggregations/coalitions formed around a discovered commonality. • Broader communication of behavior abstractions • Narrower drill-down into specific relationships • Modeling in a sparse graphical representation • Resolution: one-shot, or persistent (organization)

16. Hierarchical Representation A A to destination B A upper route A lower route A up A down A down A up

17. Top-Down Coordination

18. Top-Down Coordination

19. Top-Down Coordination

20. Top-Down Coordination

21. Top-Down Coordination

22. Top-Down Coordination

23. Technical Challenge:Management in Continuous Operations In larger, more emergent multiagent systems, the world evolves in often expected ways. Goal: Manage activities and interactions to adapt to changing circumstances and exploit ephemeral opportunities. Example Technical Foundations: dynamic belief networks, disjunctive temporal constraint networks, stalling strategies, plan repair, conditional planning. Example Strategic Direction: Multiagent Plan Management. • Distributed disjunctive temporal constraint networks • Distributed dynamic belief networks • Exploitation of graphical models for selective information propagation and processing

24. Technology Gaps Need to do: • Distributed, constrained MDPs • Real-Time behavior • Manage continuous plans • Social autonomy • Large-scale coalescing into relationship clusters • Emergent organization Can (somewhat) do: • MDPs, POMDPs • Reactive behavior • Plan for achievement • Adjustable autonomy • Relationship identification • One-shot coordination

25. Other Directions Languages and Ontologies Deception/Trust Problem Decomposition and Distribution Team Composition and Coordination Resource Allocation Multiagent Learning …