ROBOTICS COE 584 Deliberative & Hybrid Control

1. ROBOTICS COE 584 Deliberative & Hybrid Control

2. Lecture Outline • Deliberative control • Hybrid control • Types of layer organization • selection • advising • adaptation • postponement • Examples of hybrid control • AuRA, Atlantis • SSS, PRS

3. Deliberative Systems • Purely deliberative systems are considered the classical control architecture, since they were the first to be tried • In AI, classical deliberative, planner-based architectures were used for reasoning about actions in various non-physical domains, such as chess • As a result, the same architectures were applied to robotics as well

4. In the 1960’s: Shakey • In the late 1960's, the state-of-the-art in machine vision was used to process visual information on a robot called Shakey, the forerunner of many AI-inspired robotics projects. • Shakey used a classical planner as the underlying structure to decide what to do. • What is planning?

5. Planning as Search • Planning is looking ahead, searching • The goal is a state • The robot's entire state space is enumerated, and searched, from the current state to the goal state • Different paths are tried until one is found that reaches the goal • If the optimal path is desired, then all possible paths must be considered in order to find the best one

6. SPA = Planner-based • Planner-based (deliberative) architectures typically involve three generic sequential steps or functional modules: • 1) sensing (S) • 2) planning (P) • 3) acting (A), executing the plan • Thus, they are called SPA architectures • SPA has serious drawbacks • What are they?

7. Problem 1: Time • Complex state spaces: • very slow plan generation • Dynamic worlds: • out of date plans (latency)

8. Problem 2: Space • Representation of state space may be very large • Search tree (intermediate plan data) may be very large • Modern machines have virtual memory (page to disk), but swapping is very slow

9. Problem 3: Representation • Representation for planning has two parts: • Knowing the state of the world • Predicting the outcome of actions • State representation assumed to be: • complete • accurate • current • predictable

10. Problem 3: Representation • Sensors have: • noise • inaccuracies • aliasing (partial observability) • Effectors are: • unpredictable • unreliable • None of the assumptions are valid!

11. Problem 4: Execution • Execution is assumed to be: • sequential • reliable • unique (one actor) • But: • blind execution of long sequences of unreliable actions will fail • E.g., p(success | 1 action) = 0.90 • => p(success | 10 actions) = 0.35

12. Deliberative Summary • In short, deliberative (SPA) approaches: • require search (slow) • require representations (hard) • encourage open-loop execution (dangerous)

13. Opposition to SPA • As a consequence, much opposition from real robot practitioners mounted against SPA architectures • In the early/mid 1980's alternatives were proposed • reactive systems • hybrid systems • What happened to purely deliberative systems?

14. Role of Pure Deliberation • Pure deliberation is alive and well in other domains, like game playing (chess, go, etc.) and other static worlds with plenty of time to plan

15. Planners Live On in Robotics • The SPA approach has not been abandoned, it has been expanded • Given the two fundamental problems with purely deliberative approaches, we can augment them: • search/planning is slow, so save/cache important and/or urgent decisions; • open-loop plan execution is bad, use closed-loop feedback, and be ready to respond or re-plan when the plan fails.

16. Reusing Plans • Some frequently useful planned decisions may need to be reused, so to avoid planning, an intermediate layer may cache and look those up • These can be • intermediate-level actions (ILAs) • macro operators: plans compiled into more general operators for future use

17. Universal Plans • Suppose for a given problem, all possible plans are generated for all possible situations in advance, and stored • If for each situation a robot has a pre-existing optimal plan, it can react optimally, be reactive and optimal • It has a universal plan • (These are complete reactive mappings)

18. Viability of Universal Plans • A system with a universal plan is reactive; the planning is done at compile-time, not at run-time • Universal plans are not viable in most domains, because they require that: • the world must be deterministic • the world must not change • the goals must not change • The world is too complex (state space is too large)

19. Situated Automata • A formal notion of finite state machines whose inputs are connected to sensors and whose outputs are connected to effectors are called situated automata. • Situated means existing in and interacting with a complex world, and automata is the formal name for FSMs (formally: finite state automata). • Situated automata are used to create reactive principled control systems.

20. Control w/ Situated Automata • Situated automata can be constructed in two basic ways: • By hand (i.e., the designer puts FSMs together), as in the Subsumption Architecture). • By pre-compiling a complete plan (similar to Universal Plans, but reduced down to circuits of FSMs). This requires the use of a special programming language that implements the right semantics and compiles down into FSM circuitry, as Rex and Gapps.

21. Domain Knowledge • A key advantage of pre-compiled systems is that domain knowledge, i.e., information that the designer has about the environment, the robot, and the task, can be embedded into the system in a principled way • Then, the system is compiled into a reactive circuit, so the knowledge does not have to be reasoned about (or planned with) explicitly, in real-time

22. Disadvantages • A key disadvantage of pre-compiled systems is that it quickly becomes prohibitively large to enumerate the state space of a real robot, and thus pre-compiling generally does not scale up to complex systems • Another disadvantage is common to compiled or hard-wired systems: the result is not flexible in the presence of changing environments, tasks or goals

23. Inventing Hybrid Control • The basic idea is simple: we want the best of both worlds (if possible) • The goal is to combine closed-loop and open-loop execution • That means to combine reactive and deliberative control • This implies combining the different time-scales and representations • This mix is called hybrid control

24. Organizing Hybrid Systems • A hybrid system typically consists of three components: • a reactive layer • a planner • a layer that puts the two together • Hybrid architectures are often called three-layer architectures (TLA) • The planner and the reactive system are both standard, as we have covered them so far

25. The Magic Middle • The middle layer has a hard job: • 1) compensate for the limitations of both the planner and the reactive system • 2) reconcile their different time-scales • 3) deal with their different representations • 4) reconcile any contradictory commands between the two • This is the challenge of hybrid systems

26. Interaction of Layers • Hierarchical integration • Planning guides reaction • Coupled planning & reacting

27. Dynamic Re-planning • Reaction can influence planning • Any "important" changes discovered by the low-level controller are passed back to the planner in a way that the planner can use to re-plan • The planner is interrupted when even a partial answer is needed in real-time • The reactive controller (and thus the robot) is stopped if it must wait for the planner to tell it where to go.

28. Planner-Driven Reaction • Planning can influence reaction • Any "important" optimizations the planner discovers are passed down to the reactive controller • The planner’s suggestions are used if they are possible and safe • Who has priority, planner or reactor?

29. Types of Interaction • Selection: Planning is viewed as configuration • Advising: Planning is viewed as advice giving • Adaptation: Planning is viewed as adaptation of controller • Postponing: Planning is viewed as a least commitment process

30. Selection Example: AuRA • R. Arkin (1986) • Planning is viewed as configuration • Initial A* planner integrated with schema-based controller • Provides modularity, flexibility, and adaptability

31. AuRA Schematic

32. Advising Example: Atlantis • E. Gat (1991) (JPL) • Three layers: controller, sequencer, deliberator • Asynchronous, heterogeneous: reactivity and deliberation • Implemented in ALFA (A Language for Action) • Planning as advice giving, not decree • Notion of cognizant failure • Tested on NASA rovers Rocky 4

33. Atlantis Schematic

34. Adaptation Example: Planner-Reactor • D. Lyons (1992) • Continuous modification of a reactive control system • Planning is a form of reactor adaptation • Adaptation is on-line rather than off-line deliberation • Planning is used to remove performance errors when they occur • Uses a particular underlying mathematical model called a process algebra • Tested in both assembly cell and grasp planning

35. Planner-Reactor Architecture GOALS PLANNER ADAPTION REACTOR ACTION WORLD REACTIONS PERCEPTIONS PERCEPTION SENSING

36. Postponing Example: PRS • PRS = Procedural Reasoning System • Georgeff and A. Lansky (1987) • Least commitment via plan elaboration postponement • Tested on SRI Flakey

37. Flakey the robot

38. PRS Schematic

39. Another Example: SSS • J. Connell (1992) • SSS = Servo Subsumption Symbolic • 3 layers: servo, subsumption, symbolic • World models are a convenience, not a necessity • Symbolic: where-to-next (discrete time) • Subsumption: where-to-go-now • Servo: making it go (continuous time) • Tested on TJ

40. SSS Implementation: T J

41. More Examples • SOMASS hybrid assembly system • C. Malcolm and T. Smithers (Edinburgh U.) • cognitive/subcognitive components • planning as configuration • Agent architecture • B. Hayes-Roth (Stanford) • physical and cognitive levels • functional boundary blurry • Multi-valued logic • Saffiotti, Konolige, Ruspini (SRI)

42. Even More Examples • Supervenience • L. Spector (1992, U. of Maryland) • Multiple levels of abstraction • Teleo-reactive agent architecture • Benson and N. Nilsson (1995, Stanford) • Planning yields TR operator tree • Reactive Deliberation • M. Sahota (1993, U. of British Columbia) • Robosoccer

43. Still More Examples • Theoagent • T. Mitchell (CMU, 1990) • Reacts when it can plans when it must • Emphasis on learning • Generic Robot Architecture • Noreils and Chatila (1995, France) • 3 levels: planning, control system, functional • Dynamical Systems Approach • Schoner and Dose (1992) • Planning is selecting and parameterizing behavioral fields • Behaviors use vector summation

44. And Still More Examples • Integrated path planning and dynamic steering control • Krogh and C. Thorpe (1986, CMU) • Relaxation over grid-based model with potential fields controller • Planner generated waypoints for controller • Many others (including several for UUVs)

45. Hybrids Everywhere? • Hybrid systems are the most popular alternative for single-robot control • Behavior-based systems are not used by quite as many researchers, but have more specialized niches (e.g., multi-robot systems) and more practical applications

46. Textbook Readings • MM 13, 15