1 / 31

Modelling Synergies in Large Human-Machine Networked Systems cs.cmu/~softagents/muri_7

Modelling Synergies in Large Human-Machine Networked Systems www.cs.cmu.edu/~softagents/muri_7. Research Team: CMU: K. Sycara (PI), C. Lebiere, P. Scerri Cornell: M. Campbell George Mason: R. Parasuraman MIT: J. How, M. Cummings U of Pittsburgh: M. Lewis. Motivation and Research Goals.

hicksr
Download Presentation

Modelling Synergies in Large Human-Machine Networked Systems cs.cmu/~softagents/muri_7

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Modelling Synergies in Large Human-Machine Networked Systemswww.cs.cmu.edu/~softagents/muri_7 Research Team: CMU: K. Sycara (PI), C. Lebiere, P. Scerri Cornell: M. Campbell George Mason: R. Parasuraman MIT: J. How, M. Cummings U of Pittsburgh: M. Lewis

  2. Motivation and Research Goals Motivation and Importance of the Problem: Large scale H-M systems (eg large scale emergency response, NCW) offer new opportunities and possible unintended consequences, system vulnerabilities, or unanticipated changes to human roles. Objective:Develop validated theories andtechniques to understand, model and predict behavior of large-scale, networked human-machine (H-M) systems model human decision making efficiency in such H-M systems predict aggregate system behavior investigate adaptive automation to enhance human and overall system performance Second Year Review

  3. State of the Art Limitations State of the Art: Theory and computational models of collaboration of machine agents Computational models of single human information processing Single human-automation studies Psychological theories of small size human teamwork BUT No effective models of large scale interactions of human-agent systems, especially intelligent information processing agents and decision-making humans Human-in-the-loop experiments to gather data Abstract large scale models to capture important effects but still remain manageable No cognitive models of multiple interacting humans Cognitive models for single human must be extended to account for impacts of interaction. No established methods for effective model abstraction A process of abstraction and validation needs to be designed to promote confidence in the abstracted models. Second Year Review

  4. Methodology Multi-level modeling methodology (increasing abstraction) to allow studying trade-offs between model tractability and fidelity as complexity and scale increase Allow model study and validation Human-in-the-loop (level 1): perform controlled experiments with human decision-makers in small group settings The resulting data will be used to develop, constrain and validate high-fidelity cognitive models of the various operational and decision-making roles. High-fidelity models (level 2):will be validated in the same small-group settings as the human experiments, then used to generate predictions in larger but still tractable simulations (on the order of dozens of entities) Results from those simulation runs will be used to develop interacting and more abstracted agent and cognitive models in large numbers. Second Year Review

  5. Methodology Large scale simulations (level 3): further abstraction of cognitive models interacting with automated agents. Data from these simulations will be validated against level 1 and 2 data. Scale on the order of hundreds of entities. The dynamics of the level 3 simulations will be abstracted in models of high abstractions(level 4)(e.g. statistical mechanics models) that can be used to describe the dynamics of systems involving thousands of entities. Data from these simulations with be validated against data from lower levels. Second Year Review

  6. Technical Approach • Develop techniques for: • human control of large robotic teams • task allocation and scheduling in H-M teams • operator decision making under high cognitive loads in H-M systems • emergent dynamics of large H-M systems • Examine the effects of complexity on system performance, by varying such factors as: • the number and types of UVs • the amount of sensor information • network topology • number of nodes in the network. • Research Domains: Human interaction with unmanned vehicles (UVs) – this domain incorporates the aspects of information fusion, interaction with automation, coordination, planning, and command, under uncertainty • Study scaling and integration of cognitive model functionality into large systems of machine agents Second Year Review

  7. Research Issues What characteristics of human cognition and machine processing must be abstracted to enable large scale modeling? Adaptivity and learning? Understand scaling properties of cognitive performance across time and varying cognitive loads Understand dynamics of large scale networked systems What are vulnerabilities or beneficial effects? How to develop decentralized local control regimes to achieve overall performance improvement? Second Year Review

  8. Research Issues Understand interaction between humans and machines How can teams of people control teams of UVs of increasing size? What is the density of UVs a human(s) can control? What kinds of command are possible for a particular density? As the complexity of the networked system increases, e,g, scaling up in numbers, system performance may degrade because the cognitive capacities of individual human supervisors will be exceeded. Simply increasing the number of operators may not solve the problem due to increased coordination demands between hybrid team members Second Year Review

  9. Scaling of cognitive performance and workload with complexity Level 1,3 Level 1,2 Level 4 Level 2 Cornell MIT GMU Pitt CMU Robotics CMU Psychology Task allocation among agents Probabilistic models of human decision-making in network situations Level 1-2.5 Level 1-3 Level 1 Level 1,3 Decentralized control search and planning Level 1,2 Level 1,2 Level 1,2 Level 1-3 Level 1-2.5 Level 1 Level 3 Level 3-4 Level 2 Level 2 Level 2 Level 2,3 Level 1,2 Information fusion Network performance as a function of topology Communication, evolution, language Adaptive automation

  10. Real & Simulated Testbeds Second Year Review

  11. Accomplishments • ACT-UP: a new human cognitive modeling toolbox: • Rapid prototyping (programming language) • Large models of complex behavior • High fidelity, well evaluated core (ACT-R architecture) • ACT-UP used to control robots in USARSIM search and rescue • Path planning waypoint data fit human experiments • Scaling multiple operators’ span of control for multiple robots • Control complexity scheme • Studies of appropriate control for different complexity classes • Gesture based interface for multi robot team control • Human-Agent Information Fusion Second Year Review

  12. Accomplishments • Task allocation and scheduling algorithms for large H-M agents in uncertain environments to • Ensure spatial and temporal coordination • Improve mission efficiency and cost, reduce operator over-utilization • Investigation of large scale network system dynamics • Information avalanches and scale invariant dynamics • DACOR: distributed algorithm for local system adaptation to optimize global performance in non-linear dynamics • Generalization and model for inducing scale invariant dynamics • Geo Game: experimental platform for studying behavior of large number of networked humans • initial comparison with agent simulations • Adaptive operator aiding during periods of high cognitive load Second Year Review

  13. Pitt-CMU Robotics: collaborative experiments in multirobot/multihuman control Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 1,2 Level 1-2.5 Level 3 Level 2 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Level 1,2 Information fusion Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  14. Multi human/robot Control Experiments Second Year Review

  15. MIT-Pitt: using multirobot control data (Pitt) as data set for validating queuing control model of operator (MIT) Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 1,2 Level 1-2.5 Level 3 ? Level 2 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Information fusion Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  16. Meta-Analysis Pitt/MIT Second Year Review

  17. Pitt-CMU Psychology: fitting multirobot control data (Pitt) with ACT-UP cognitive model (CMU-Psy) Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 1,2 Level 1-2.5 Level 3 Level 2 Level 1,2 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Information fusion Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  18. Cognitive model of spatial path planning in MrCS Data from multi robot control Experiments fit to model of visually Guide path planning Second Year Review

  19. Cornell-MIT: using task & trajectory planning (MIT) with human & sensor fusion (Cornell) Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1,2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 1,2 Level 1-2.5 Level 3 Level 2 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Level 1,2 Information fusion Level 1,2 Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  20. Cornell/MIT Collaboration 20 Second Year Review Second Year Review 10/23/2019

  21. Cornell-GMU: analyzing human subject data from DDD (GMU) using Control models (Cornell) Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 3,4 Level 1-2.5 Level 3 Level 2 Level 1,3 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Level 1,2 Information fusion Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  22. Cornell/GMU Collaboration 22 Second Year Review Second Year Review 10/23/2019

  23. CMU Psychology & Robotics, Pitt: Geo-game, large scale human network experiment Cornell MIT GMU Pitt CMU Robotics CMU Psychology Scaling of cognitive performance and workload Level 1,3 Level 1,2 Level 1 Level 2 Level 1-2.5 Level 1-3 Level 1 Level 1,3 Task allocation among humans/agents Probabilistic models of human decision-making in network situations Level 1,2 Level 1-2.5 Level 3 Level 2 Level 1-3 Decentralized control search and planning Level 1,2 Level 2 Level 1,2 Information fusion Level 1,2 Level 1-3 Level 1,3, 4 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1

  24. Geo Game Goal: Geo Game as an experimental platform to: • Study behavior of large number of networked humans • Trade-off: communication vs. task execution • Formation of coalitions • Reciprocity • Effects of network interactions on cognition • Current task: cooperative foraging • Subjects can communicate only with network neighbors • Other variants: trading agents, varied information types (stochastic, graded, discrete, etc.), other networks (trees, adversarial networks) • Develop computational models with similar interactions to understand dynamics and behavior Second Year Review

  25. The Geo Game Second Year Review

  26. Metrics • 81 publications in journals, books and conferences • Best paper award: • Reitter and Lebiere: “A sub-symbolic and visual model of spatial path planning”, BRIMS 2009 • Second place for best paper award: • Glinton, Scerri, Sycara: “Exploiting Scale Invariant Dynamics for Efficient Information Propagation in Large Teams”, AAMAS 2010 • 3 keynote talks • 10 graduate students • 4 postdoctoral fellows Second Year Review

  27. Deliverables • Validated methodology for the study of complex human-machine systems • Analytical results, for smaller scale problems that can serve as “islands of validation” for the larger scale simulations • Results on system performance for various phenomena of interest, e.g. unpredictable system behavior and vulnerability • Techniques for principled abstraction of human models and their behavior properties and effect on system performance. Second Year Review

  28. Research Significance Expected Results Understand the underlying principles of developing H-M teams that operate efficiently together in multiple dynamic uncertain environments Understand the tradeoff between modeling fidelity and computational tractability of H-M systems Identification of potential bottlenecks, challenges to stability, and obstacles to human control in large H-M systems Breakthrough Opportunity: Enable the prediction of nonlinear behavior of large scale networked H-M complex systems; Recognition and mitigation of their harmful interactions and effects; Design of large H-M complex systems with desirable overall behavior; Second Year Review

  29. Potential Applications to DoD • NCW offers both new opportunities and the possibility of unintended consequences or unanticipated changes to human roles. The proposed research explores these issues through: • mathematical models, • simulation • cognitive models • The research will provide • Identification of potential bottlenecks, challenges to stability, and obstacles to human control • Solutions to these problems before they are encountered in the field Second Year Review

  30. Plans for Next Year • Experiments with networked humans • Small number of human nodes that fuse probabilistic data in indoor distributed mapping • Large number of humans on the Internet in information fusion and decision making tasks • Experiments with hybrid human agent teams in foraging and pursuit/evasion tasks • Experiments with team size invariant displays for large UV teams • Experiments with multiple operator, multiple UVs in different operator team configurations • Continue work on ACT-UP • Experiments with ACT-R and ACT-UP cognitive models interacting with UVs • Extension of human/sensor data fusion experiments Second Year Review

  31. Questions? Second Year Review

More Related