Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments

1. Introduction to Robots and Multi-Robot SystemsAgents in Physical and Virtual Environments Lecture 2: Reactive Systems Gal A. Kaminka galk@cs.biu.ac.il

2. Previously, on Robots … • Robots are agents in physical or virtual environments • Persistent, situated, responsive • Environments have different characteristics • Static/dynamic, accessible?, deterministic?, …. • Since we are lazy, we want robots to do things for us • Robots must consider task when deciding what to do • Action-selection problem: What to do now in service of the task?

3. Physical Environments • Dynamic • Non-deterministic • Inaccessible • Continuous

4. A Crash Course in AI Planning • Planning: An approach to action-selection problem • Very long history, since the very beginning of AI • 1971, seminal paper: STRIPS (Fikes and Nilsson) • Still cited and taught today, despite much progress • STRIPS originally developed for SRI robot, Shakey • This is ironic • Later on, STRIPS planning was rejected for robot control

5. A B C A D B C AI Planning Definition: State { On(A,table), On(B,table), On(C,table), In-hand(nil) } { On(A,B), On(B,table), On(C,table), In-hand(D) }

6. AI Planning Definition: Operators • Operators change state of the world • Preconditions, add-list, delete-list • Pick-A(?x) • Preconditions: On(A, ?x), In-hand(nil) • Add: In-hand(A) • Delete: In-hand(nil), On(A, ?x) • Put-A(?y) • Preconditions: In-hand(A), not On(?x, ?y) • Add: On(A,?y), In-hand(nil) • Delete: In-hand(A)

7. STRIPS Planning • Given: • Initial state of the world • Operators • Goal state • Produce: • Plan: Ordered list of instantiated operators • Will change the world from initial state to goal state

8. Initial State After Put-B(A) After Pick-B(table) B B C A B C A C A C C B B A A After Pick-C(table) Goal State Planning Example After Put-C(B)

9. Planning on robots • Sense initial state using sensors • Create a full plan given goal state (given task) • Feed plan, step-by-step to motors • No need to sense again What’s wrong with this? (Hint: Think about Schoppers’ paper)

10. Deliberative Control Deliberative: • Has internal state (typically a model of the world) • Uses this internal state to make decisions • Decisions made between alternatives Sense Model Think Act

11. When plans goes wrong • Dynamic environment • State changes even if no operator applied • Non-deterministic • State changes not according to operator specs • Inaccessible • Cannot sense entire state of the world • Continuous • Predicate-based description of world is discrete

12. Reactive control • Reactive: • No internal state • Direct connection from sensors to actions • S-R (stimulus response) systems • No choices, no alternatives Sense Hard Wiring Act

13. Universal Planning • Have a plan ready for any possible contingency • Scouts: Be prepared!  • From any initial state, know how to get to goal state • Input: Operators, goal state • Do not need to give initial state • Output: Decision tree • What operator to take, depending on environment state • Not a single ordered list of operators

14. Initial State After Put-B(A) After Pick-B(table) B B C A B C A C A C C B B A A After Pick-C(table) Goal State Universal planning algorithm

15. Robot Control AlgorithmUsing Universal Planning • Robot given task (goal, operators) • Uses universal planner to create universal plan • Robot senses environment • Goal state reached? • No: Execute operator according to decision tree • Yes (keep persistency)

16. Advantages of Universal Planning • Guaranteed to use optimal (shortest) plan to goal • A very good thing! • Optimal solution to action selection problem • Robust to failures • Robust in dynamic and non-deterministic domains

17. Problems with Universal Planning • Assumes accessibility • Assumes perfect sensors • Assumes discrete actions (operators)

18. Universal plan as mapping sensors to actions • Universal plan can be viewed as a function • Sensor readings to actions u: SA • Essentially a table: For each state, give action • Schoppers uses a decision-tree representation

19. Problems: Planning Time • What is the planning time? • Planningtime grows with the number of states • Since we have to enumerate operator for every state • What is the number of states in an environment? Worst case: All possible combinations of sensor readings (state predicates)

20. X2 X1 X2 X1 Put Put Put Put Pick Pick Pick Pick Problems: Universal Plan Size • Plan size grows with the number of possible states • “Curse of dimensionality” Pick Put X1 Put Pick X2

21. Problems: Stupid executioner • Schoppers: • Baby goes around knocking blocks around? • Ginsberg: • What if baby repeatedly knocks down the same block? • Universal plans may get into cycles • This is because no deliberation is done • Universal planner relies on simple executioner • Sense, consult table, act • Same as regular planner – except for sensing

22. שאלות?

23. Brooks’ Subsumption Architecture Multiple levels of control: Behaviors Plan changes Identify objects Monitor Change Map Explore Wander Avoid Object

24. Why does this work? • It breaks the ideal universal plan into behaviors • avoids the curse of dimensionality • Behaviors (levels) interact to generate behavior • Note that programmer is responsible for task-oriented design • Goes both below and above universal plans • Hand programmed: approximate plan • Not automatically generated

25. Subsuming Layers • How to make sure overall output is coherent? • e.g., avoid object is in conflict with explore • Subsumption hierarchy: Higher levels modify lower Map Explore Wander Avoid Object

26. Coherence using subsumption Key principle Higher layers control input/outputs of lower layers In practice: • Can be difficult to design • a lot depends on programmer • Single module per layer can be restrictive • Increasingly challenging to control at higher levels

27. Irony • Ginsberg’s article pretty much killed universal planning • Though occasional papers still published • Reactive control very popular in practice • But due to theory problem, no more automated planners! • So we get lots of reactive plans, but no planning

28. Irony again • Ginsberg was right: • Approximating universal plan is possible • Tends to be useful only in fairly low level locomotion control • Approximation is what Brooks had done • Which is why he often gets the credit for the revolution

29. Starting next week Behavior-Based Control: Expanding on Subsumption