June 10, 2002

1. June 10, 2002 Adaptive Coordinated Control in 3-D Dynamic Battlefield

2. Adaptive Coordinated Control of Intelligent Multi-Agent Teams Shankar Sastry, Ruzena Bajcsy, Laurent El Ghaoui, Mike Jordan, Jitendra Malik, Stuart Russell, Pravin Varaiya (Berkeley) Vijay Kumar, Kostas Danillidis, James Ostrowski, George Pappas, C. J. Taylor (Penn) Howie Choset, Alfred Rizzi, Charles Thorpe (CMU) Adaptive Coordinated Control in 3-D Dynamic Battlefield

3. Background • ARO-MURI “Integrated Approach to Intelligent Systems” 1996-2001. Partners: Berkeley, Stanford, Cornell. Highlights: • Creation of Field of Hybrid Systems: Foundations, Methods, Analysis, Control • Vision Based Navigation and Control • Major Force Driving Bayesian Networks, Graphical Models, Dynamical Probabilisitic Networks, Learning, Rapproachment of AI and Control Adaptive Coordinated Control in 3-D Dynamic Battlefield

4. Integrated Approach to Intelligent Systems • Disciplinary Evolution • Control Theory • Optimal control, linear control, nonlinear control, adaptive control, stochastic control • Mathematics: differential equations • Artificial Intelligence • Reasoning, adaptation, neural networks, natural language, expert systems • Mathematics: formal logic • Computational Neuroscience & Cognitive Science • Sensing, vision, taction, olfaction neural networks • Mathematics (recently developed) Adaptive Coordinated Control in 3-D Dynamic Battlefield

5. Central Control Paradigm. that is sensors and actuators interacting locally, breaks down when dealing with distributed systems due to Complexity & scale Necessity of “tight” or “optimal” operations Key Characteristics of Distributed Intelligent Systems Hierarchical or modular to control complexity Globally organized emergent behavior Robust, adaptive and fault tolerant, and degraded modes of operation Architectural organization involving the use of compositionality Why Hierarchical Hybrid Control? Adaptive Coordinated Control in 3-D Dynamic Battlefield

6. Why Hybrid Hierarchical Control? • Intelligence Augmentation for Human-Centered Systems • Autonomous Intelligence Why integrative? Due to: - the need to merge sensor fusion and hierarchies of sensing with actuation across many agents, with desired emergent behavior - the need to merge logical decision making and continuous action - the need reconcile the need for safety of individual agents with collective optimality Control, artificial intelligence and cognitive neuroscience deal with continuous action, logical reasoning and human/machine understanding, respectively Adaptive Coordinated Control in 3-D Dynamic Battlefield

7. Technology Drivers: Semi-Autonomous Multi-Agent Systems The need for a theoretical framework for an integrative approach arises from advances in computation, communication, intelligent materials, visualization and other technologies which make it possible to expect more from a multi-agent system than from a centralized control framework. • Distributed Command and Control • Distributed Communication Systems • Distributed Power Systems • Intelligent Vehicle Highway Systems • Air Traffic Management Systems • Intelligent Telemedical Systems • Intelligent Manufacturing Systems • Unmanned Aerial Vehicle Networks • Mobile Offshore Bases Adaptive Coordinated Control in 3-D Dynamic Battlefield

8. Theoretical Underpinnings • Architectural Design for Multi-Agent Systems • Hybrid Systems • Centralization for optimality • Decentralization for safety, reliability and speed of response • Perception Systems Sharing Many Representations • Hierarchical aggregation • Wide-area surveillance • Low-level perception • Frameworks for Representing and Reasoning with Uncertainty • Incorporation of Learning, Adaptation and Fault Tolerance • Parametric uncertainty with update and adaptation at the continuous levels, learning of new “logical entities” --reinforcement learning at the logical levels and metal-learning for redefining architecture Adaptive Coordinated Control in 3-D Dynamic Battlefield

9. What Are Hybrid Systems? • Dynamical systems with interacting continuous and discrete dynamics Adaptive Coordinated Control in 3-D Dynamic Battlefield

10. Why Hybrid Systems? • Modeling abstraction of • Continuous systems with phased operation (e.g. walking robots, mechanical systems with collisions, circuits with diodes) • Continuous systems controlled by discrete inputs (e.g. switches, valves, digital computers) • Coordinating processes (multi-agent systems) • Important in applications • Hardware verification/CAD, real time software • Manufacturing, communication networks, multimedia • Large scale, multi-agent systems • Automated Highway Systems (AHS) • Air Traffic Management Systems (ATM) • Uninhabited Aerial Vehicles (UAV), Power Networks Adaptive Coordinated Control in 3-D Dynamic Battlefield

11. Control Theory Computer Science Models of computation Control of individual agents Communication models Continuous models Discrete event systems Differential equations Hybrid Systems Framework Adaptive Coordinated Control in 3-D Dynamic Battlefield

12. Control Challenges • Large number of semiautonomous agents • Coordinate to • Make efficient use of common resource • Achieve a common goal • Individual agents have various modes of operation • Agents optimize locally, coordinate to resolve conflicts • System architecture is hierarchical and distributed • Safety critical systems Challenge: Develop models, analysis, and synthesis tools for designing and verifying the safety of multi-agent systems Adaptive Coordinated Control in 3-D Dynamic Battlefield

13. Taking Stock in Hybrid Systems • Hybrid Systems and Control: established as a discipline, taught to undergrads, grads. Monographs, textbooks being written by all co-PIs: Lee and Varaiya, Henzinger and Alur, Lygeros, Tomlin and Sastry. Workshop on Hybrid Systems established (first was in Berkeley in 1998). Special Issues in IEEE Proceedings, Systems and Control Letters, Automatica, IEEE Transactions on Automatic Control, … • Software: Programming languages, tools and frameworks for Simulation and Control: Ptolemy II, Giotto, Massaccio all developed. Ongoing work on verification tools. • Hardware: in the loop demonstrations on the local UAVs, formation flying to follow. • Embedded software: EMSOFT established, new IEEE Proceedings Special Issue 2003. Adaptive Coordinated Control in 3-D Dynamic Battlefield

14. UCB/UCSF Laparoscopic Telesurgical Workstation Adaptive Coordinated Control in 3-D Dynamic Battlefield

15. ANIMAL LAB TRIALS 1998 Adaptive Coordinated Control in 3-D Dynamic Battlefield

16. Suturing with Unimanual System, 1998 Adaptive Coordinated Control in 3-D Dynamic Battlefield

17. Berkeley BEAR Fleet: Ursa Maxima 1 Based on Yamaha RMAX industrial helicopter Integrated Nav/Comm Module Length: 3.63m Width:0.72m Height: 1.08m Dry Weight: 58 kg Payload: 30kg Engine Output: 21 hp Rotor Diameter: 3.115m Flight & system operation time: 60 min Adaptive Coordinated Control in 3-D Dynamic Battlefield

18. Flight Control System Experiments Landing scenario with SAS (Dec 1999) Position+Heading Lock (Dec 1999) Position+Heading Lock (May 2000) Attitude control with mu-syn (July 2000) Adaptive Coordinated Control in 3-D Dynamic Battlefield

19. Pursuit-Evasion Game Experiment Setup Waypoint Command Pursuer: UAV Current Position, Vehicle Stats Evader location detected by Vision system Ground Command Post Current Position, Vehicle Stats Evader: UGV Adaptive Coordinated Control in 3-D Dynamic Battlefield

20. Pursuit-Evasion Game Experiment • PEG with four UGVs • Global-Max pursuit policy • Simulated camera view • (radius 7.5m with 50degree conic view) • Pursuer=0.3m/s Evader=0.5m/s MAX Adaptive Coordinated Control in 3-D Dynamic Battlefield

21. Experimental Results: Pursuit Evasion Games with 4UGVs (Spring’ 01) Adaptive Coordinated Control in 3-D Dynamic Battlefield

22. Pursuit-Evasion Game Experiment • PEG with four UGVs and a UAV • Global-Max pursuit policy • Simulated camera view • (radius 7.5m with 50degree conic view) • Pursuer=0.3m/s Evader=0.5m/s MAX Adaptive Coordinated Control in 3-D Dynamic Battlefield

23. Experimental Results: Pursuit Evasion Games with 4UGVs and 1 UAV (Spring01) Adaptive Coordinated Control in 3-D Dynamic Battlefield

24. What is Different Today? • The world and national security threats are different: mobile operations in urban terrain, hostage rescue, anti terrorism operations, homeland protection. • Use of robotic and mixed initiative forces, the need for coordination of manned and unmanned forces • The need for dynamic strategies and tactics for dealing with a determined and flexible adversary. • Exploitation of the 3rd dimension by organic UAVs Adaptive Coordinated Control in 3-D Dynamic Battlefield

25. New Technical Innovations • Control of the 3 D Digital battlefield: need to use 3rd dimension, aerial forces, robotic and mixed initiative forces, untethered communications • Adaptive Coordinated Control of Multiple Agents: reconfiguration of teams dynamically in response to adversarial action • Intelligent coordination of multiple agents: ability to discover intent and reconfigure strategies adaptively Adaptive Coordinated Control in 3-D Dynamic Battlefield

26. Intellectual Organization:Thrust Areas • Architecture Design for Adaptive, Dynamic Planning • Integration of Rich Multi-Sensor Information into Virtual Environments incorporating human intervention • Handling Uncertainty and Adversarial Intent in Adaptive Planning Adaptive Coordinated Control in 3-D Dynamic Battlefield

27. Challenge Scenarios • Reconaissance and Intelligence: robotic ranger force for scouting fixed area for time critical targets • Mixed Initiative Engagement in urban environments using micro-UAVs, UGVs. Emphasis on immersive environments for deploying • Recognition and Tracking of Unfriendlies: emphasis on networked vision for tracking. Adaptive Coordinated Control in 3-D Dynamic Battlefield

28. Hierarchical Architectures for Dynamic Adaptive Planning • Progess to date in hierarchical architectures for decision making in “normal” modes of operation. Main emphasis here will be on replanning in “fault” or “degraded” modes of operation including deviations from hierarchical operation. • Key technical issues: • Abstractions of Hybrid Systems for Architecture Design • Hierarchical abstractions • Assume-guarantee reasoning for abstractions Adaptive Coordinated Control in 3-D Dynamic Battlefield

29. Thrust I continued • Control of Hybrid Systems • Numerical Solutions for Controller Synthesis • Hierarchical Solutions of Synthesis Procedures • Liveness and other acceptance conditions • Controller Libraries • Many world semantics and hierarchy semantics • Modal decomposition • Exceptions • Team and Task Allocation Adaptive Coordinated Control in 3-D Dynamic Battlefield

30. Integration of Multi-Sensor Information Into Virtual Environments • Adaptive Hierarchial Networks for Acquiring and providing information • Networked sensors • Bandwidth utilitzation • Extraction of 3 D Models from Distributed Sensors • 3 D models from video data • Integration of real and virtual environments • Environments for Human Intervention & Decision Making • Situational awareness • Display of uncertain data • Triaging of data for decision making Adaptive Coordinated Control in 3-D Dynamic Battlefield

31. Smart Dust, Dot Motes, MICA Motes Dot motes, MICA motes and smart dust Adaptive Coordinated Control in 3-D Dynamic Battlefield

32. Tiny OS (TOS) Jason Hill, Robert Szewczyk, Alec Woo, David Culler • TinyOS • Ad hoc networking Adaptive Coordinated Control in 3-D Dynamic Battlefield

33. Last 2 of 6 motes are dropped from MAV Adaptive Coordinated Control in 3-D Dynamic Battlefield

34. Field of wireless sensor nodes • Ad hoc, rather than engineered placement • At least two potential modes of observation • Acoustic, magnetic, RF Adaptive Coordinated Control in 3-D Dynamic Battlefield

35. Subset of more powerful assets • Gateway nodes with pan-tilt camera • Limited instantaneous field of view Adaptive Coordinated Control in 3-D Dynamic Battlefield

36. Set of objects moving through Adaptive Coordinated Control in 3-D Dynamic Battlefield

37. Track a distinguished object Adaptive Coordinated Control in 3-D Dynamic Battlefield

38. Many interesting problems • Targeting of the cameras so as to have objects of interest in the field of view • Collaborate between field of nodes and platform to perform ranging and localization to create coordinate system • Building of a routing structures between field nodes and higher-level resources • Targeting of high-level assets • Sensors guide video assets in real time • Video assets refine sensor-based estimate • Network resources focused on region of importance Adaptive Coordinated Control in 3-D Dynamic Battlefield

39. Abstraction of Sensorwebs • Properties of general sensor nodes are described by • sensing range, confidence on the sensed data • memory, computation capability, clock skew • Communication range, bandwidth, time delay, transmission loss • broadcasting methods (periodic or event-based) • To apply sensor nodes for the experiments with UAV/UGVsintroduce super-nodes (or gateways), which can • gather information from sub-nodes ( filtering or fusion of the data from sub-nodes for partial map building) • communicate with UAV/UGVs Adaptive Coordinated Control in 3-D Dynamic Battlefield

40. Uncertainty and Adversarial Intent • Models of Uncertainty • Environmental: non deterministic and probabilistic • Adversarial • Guarantees of Success in the face of uncertainty • Decision making in the presence of uncertainty • Learning of Adversarial Strategy • Probing strategies • Games, partial information solution concepts • Adaptation to changing utility functions of adversary Adaptive Coordinated Control in 3-D Dynamic Battlefield