P-CLAIM: Agent-Oriented Language with Temporal Planning

1. An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms Dr. Muhammad Adnan Hashmi

2. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

3. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

4. What is an Agent? • An agent is a computer system capable of autonomous action in some environment in order to meet its design objectives (goals). • Achievement of goals is important for agents

5. Planning • Finding a sequence of actions that takes agent from Initial State to Goal State Initial state:clear(c),on(c,a),ontable(a),clear(b),ontable(b),handempty Goal:on(a,b), on(b,c) a c b a b c a c b b a b a b c a c c

6. Multi-Agent Planning • Multiple agents sharing the same environment • Independent planning is not sufficient • Coordination of plans is vital • Coordination of plans • Removing conflicts (negative interactions) • Utilizing help relations (positive interactions) DA DA A A DB DB B B

7. How to Program Agents? • Agent Oriented Programming (AOP) Languages allow to program intelligent and autonomous agents • Main Characteristics • Mental State: Beliefs, Goals, Commitments • Reasoning Mechanism • Capabilities, Services • Communication • Some Languages • Agent-0 [Shoham 1993], 2APL [Dastani 2008], AgentSpeak (L) [Rao 1996]

8. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

9. Problem Statement • Current AOP languages • Follow a reactive (PRS based) approach • Do not support temporal planning • Only a few support planning • Problems • Execution without planning may result in the goal failures • Agent can reach a dead end • Conflicts can arise among different agents • Actions’ duration is important • Real world actions take place over a timespan

10. Objectives • Propose a programming language that endows agents with planning skills • Has temporal planning • Deals with uncertainty of the environment • Incorporate reactivity by dealing with on the fly goals having different priorities • Propose coordination mechanisms for the plans having different priorities

11. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

12. Assumptions • Two agents α and β sharing the same environment • Agent α having higher priority (reactive) goals • Agent β having normal priority (proactive) goals • Actions have: • Preconditions • Add effects • Delete effects • Two possible conflicts among plans • Causal link threat • Parallel actions interference

13. Two Possible Conflicts • Causal Link (A1, A2, p) • Action A1 adds an effect p • Action A2 needs this effect • No action between A1 and A2 adding p • Causal Link Threat • If an action A deletes p and lies between A1 and A2, then A threatens the causal link (A1, A2, p) p A1 A2 Threat A ¬p

14. Two Possible Conflicts • Causal Link (A1, A2, p) • Action A1 adds an effect p • Action A2 needs this effect • No action between A1 and A2 adding p • Parallel Actions Interference • Actions A1 and A2 lie in parallel • Either one of them deletes the preconditions or add effects of the other A1 p A1 ¬p p A2 ¬p A2

15. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

16. Coordinated Planning Problem • Prerequisite: • Plan Pα of Agent α • Our Aim: • Compute a Plan Pβ for Agent β • Has no conflict with Pα • Avails the cooperative opportunities offered by Pα • Solution: • Non Temporal Domains  µ-SATPLAN • Temporal Domains  Coordinated-Sapa

17. SATPLAN [Kautz and Selman 2006] A Classical Planner that Finds Optimal Plans in Non-Temporal Domains

18. SATPLAN We Use GraphPlan Encoding Propositional formula in conjunctive normal form (CNF) • Planning Problem • Init State • Goal • Actions Compiler (encoding) Simplifier (polynomial inference) CNF Increment plan length If unsatisfiable CNF satisfying model Decoder Solver (SAT engine/s) Plan

19. Constructing the planning graph • Level P1: All literals from the initial state • Add an action in level Ai if all its preconditions are present in level Pi • Add a proposition in level Pi if it is the effect of some action in level Ai-1 • Maintain a set of exclusion relations to eliminate incompatible propositions and actions

20. GraphPlan Encoding • Can create such constraints for every node in the planning graph • Only involves facts and actions in the graph Pre1 Act1 Fact Pre2 Act2 Fact  Act1  Act2 Act1  Pre1  Pre2 ¬Act1  ¬Act2

21. µ-SATPLAN An Extension of SATPLAN that Computes Coordinated Plans in Non-Temporal Domains

22. Handling Causal Link Threat • While constructing the planning graph for Agent β, don’t add an action O at level Ai if • It has an effect ¬p, and • There is a causal link (Aj, Ak, p) in plan Pα, and • j ≤ i ≤ k Action O threatens Causal Link (Aj, Ak, p)

23. Handling Positive Interactions & Parallel Actions Interference • For each time step i in the plan of Agent α, create an action NoName(i) • Add(NoName(i))  All the effects added by Pαat time step i • Del(NoName(i))  All the effects deleted by Pαat time step i • Pre(NoName(i))  All the preconditions of actions of Pαat time step i • Explicitly add all the NoName actions in the planning graph of Agent β

24. Level 0 Level 1 Level 2 a0 a0 a0 NoName(0) NoName(1) a2 a2 a1 a1 a1 β2 β4 a8 a4 a4 a4 a4 β1 a5 a5 a5 β3 a7 a7 a6 a6 a6 a6 Handling Positive Interactions • Pα = {α1(0), α2(0), α3(1)} • Eff(α1) = a0, Eff(α2) = a1, Eff(α3) = a2 • Eff( NoName(0) ) = {a0,a1}, Eff( NoName(1) ) = {a2} Planning Graph of Agent β

25. Level 0 Level 1 Level 2 a0 a0 a0 NoName(0) NoName(1) a2 a2 a1 a1 a1 β2 β4 a8 a4 a4 a4 a4 β1 a5 a5 a5 β3 a7 a7 a6 a6 a6 a6 Handling Positive Interactions • Add NoName actions as unary clauses to CNF • NoName(0) • NoName(1) • Partial CNF Sentence • a8  β4 • β4  a0  a7 • a0  β2  NoName(0) • a7  β3 • β2  a5 • … Solution Problem

26. Example Plan Generated (Logistics) Pα Pβ

27. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

28. Proactive-Reactive Coordination • Prerequisite: • Reactive plan Pα of Agent α • Proactive plan Pβ of Agent β • Our Aim: • Modify plan Pβ such that: • It has no conflict with Pα • Avails the cooperative opportunities offered by Pα • Solution: • Plan Merging Algorithm

29. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D Case Study Rescue Agent : α Analyzer Agent : β • Tasks of Rescue Agent • Rescue the Victims • Tasks of Analyzer Agent • Analyze the goal cells • Call the central agent • Constraints • One agent in a cell • Hyper energy cells • Needs fuel or energy to enter • Agent should have key to open door

30. Conflict Resolution • Threat-Repair Link (A1, A2, p) • Action A1 deletes p • A2 is a subsequent action and adds p • A1 is called Threat Action • A2 is called Repair Action p B1 B2 Threat A1 -p A2 p Repair

31. Valid and Possibly Valid Time Stamps • Possibly Valid Time Slot for an action A • All preconditions are met • No parallel actions interference P[1]  a b h P[2]  b c -d P[3]  c e P[4]  e f P[5]  f g P[6]  g i P[7] • i • h g P[1]  b d -h • Valid Time Slot for an action A • All preconditions are met • No parallel actions interference • Either: • No causal link threat • Repair Action exist before the deadline P[1]  a b h P[2]  b c -d P[3]  c e P[4]  e f P[5]  f g P[6]  g i P[7] • i • h g P[2] k P[1]  b d -h  d P[3] • k h

32. Plan Merging Algorithm • Fix all the actions of Reactive Plan Pα on timeline • For every action CA of Proactive Plan • Search for the first Possibly Valid Time Slot T on timeline • Reason about the time slot T • There could be 5 cases at T

33. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D Plan Merging Algorithm Case 1: No causal link threat by CA at T • Assign Time Slot T to CA EXAMPLE • Current Action: Move(A1, A2) • Returned Time Slot: 0 - 5 • Any Threat? : No • Assign Time Slot 0 – 5 to CA

34. Plan Merging Algorithm Case 2: CA threatens a Causal Link but Repair Action exist • Assign Time Slot T to CA • Save a Possible Threat <ThreatAction, RepairAction, Deadline> EXAMPLE • Current Action: Move(A4, A5) • Time Slot: 20 - 25 • Any Threat? : Yes (Agent α needs A5 at time 40-45) • Repair Action: Move(A5, A6) • Assign Time Slot 20 - 25 to Move (A4, A5) • Save <Move(A4, A5), Move(A5, A6), 40> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

35. Plan Merging Algorithm Case 3: It is a Repair Action but can not meet a deadline of some Threat Action • Backtrack to the Threat Action,find another time stamp EXAMPLE • Current Action: Move (A8, A9) • Returned Time Slot: 50 - 55 • Any Threat?: Yes (Agent α needs A9 at 85-110) • Repair Action : Move (A9, B9) • Save <Move(A8, A9), Move(A9, B9), 85> • Next Action: AnalyzeCell (A9) • Time Slot Assigned: 55 - 70 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

36. Plan Merging Algorithm Attention • Next Action: CallCentral (A9) • Time Slot Assigned: 80 – 90 • Next Action: Move (A9, B9) • Is it a Repair Action? : Yes • Meet all deadlines?: No (Agent α needs A9 at 85) • Backtrack to action Move(A8, A9) • Find another Time Slot • New Time Slot: 110 – 115 (Valid Time Slot) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

37. Plan Merging Algorithm Case 4: All the effects of CA are already achieved WHAT TO DO? • Mark CA as redundant POST PROCESSING • Remove all redundant actions from the plan • Recursively remove all actions which achieve only the preconditions of removed action

38. Plan Merging Algorithm EXAMPLE • Current Action: OpenDoor (C11) • Returned Time Slot: 172 - 175 • Redundant(OpenDoor(C11))  true • BecauseopenedDoor(C11) is true at time 172 • When the plan is returned • Remove OpenDoor(C11) from plan • Also remove TakeKey(C11, key1) from plan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

39. Plan Merging Algorithm Case 5: Action CA’s preconditions can not be achieved • Remove action CA from the plan and compute a plan to achieve effects of CA • I = State just before CA • G = Effects (CA) • Plan should have no conflict with Reactive Plan Pβand if CA is a repair action, repair effects must meet their deadline • ReplacementPlan = Coordinated-Sapa (I, G, Pβ) • If a plan is returned, replace the removed actions with the plan • If a deadline is violated, backtrack to the threat action • If no plan possible, then remove another action CA + 1 • G = G U Effects (CA + 1) \ Pre (CA + 1) Use Coordinated-Sapa

40. Plan Merging Algorithm EXAMPLE • Current Action: TakeEnergy(B13, energy1) • Preconditions can not be achieved • Repair the plan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

41. Plan Repair Algorithm • Create a CPP by removing TakeEnergy(B13, energy1) • I = { at(β, B13), at(energy1, B13), at(energy2, B13) } • G = { hasEnergy(β, energy), at(β, B13)} • Call Coordinated-Sapa to solve this CPP • Coordinated-Sapa returns fail • Why? energy2 is also needed by Agent α 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

42. Plan Repair Algorithm • Create another CPP by removing Move(B13, A12) • I = { at(β, B13), at(energy1, B13), at(energy2, C15) } • G = { at(β, A12) } • Call Coordinated-Sapa to solve this CPP • A plan is returned to enter A12 by taking the fuel from D14 POST PROCESSING • This plan will become a replacement for both TakeEnergy(B13, energy1) and Move(B13, A12) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D

43. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

44. P-CLAIM • An AOP language having: • Cognitive aspects specific to intelligent agents • Communication primitives • Mobility primitives • Temporal planning capability • P-CLAIM Agent: • Is autonomous, intelligent and mobile • Has a mental state containing knowledge, goals, and capabilities • Is able to communicate with other agents • Entails a planning based behaviour • Achieves goals based on their priorities • Maintains the stability of the plan in the dynamic environments

45. Defining P-CLAIM Agent defineAgent agentName{ parent = null | agentName ; knowledge = null ;| {knowledge1; …; knowledgem} goals = null;| {goal1; … ; goaln} activities = null ; {activity1 … activityo} actions = {action1 … actionp} agents = null ; | {agName1, agName2, …, agNameq} }

46. Activities defineAgent agentName{ parent = null | agentName ; knowledge = null ;| {knowledge1; …; knowledgem} goals = null;| {goal1; … ; goaln} activities = null ; {activity1 … activityo} actions = {action1 … actionp} agents = null ; | {agName1, agName2, …, agNameq} } activityity = name { message = null | message ; conditions = null | condition ; do { process } effects = null ;| { effect1 ; …; effectf } }

47. Actions defineAgent agentName{ parent = null | agentName ; knowledge = null ;| {knowledge1; …; knowledgem} goals = null;| {goal1; … ; goaln} activities = null ; {activity1 … activityo} actions = {action1 … actionp} agents = null ; | {agName1, agName2, …, agNameq} } action = name { message = null | message ; conditions = null | condition ; do { process } duration = dur ; }

48. Outline • Introduction • Background • Problem Overview • Plan Coordination Mechanisms • Coordinated Planning Problem • Proactive-Reactive Coordination Problem • P-CLAIM: AOP Language supporting Temporal Planning • Language Definition • Planning Mechanism • Conclusion

49. Agent Definition to Planning (Translator) Agent Description File Knowledge Goals Activities Actions Translator (JavaCC) Initial State Goals Methods Operators Problem File Domain File Planner

50. Agent Life Cycle