1 / 31

David C. Wilkins Center for Study of Language and Expertise Stanford University David Fried

Multi-agent architectures that facilitate apprenticeship learning for real-time decision making: Minerva and Gerona. David C. Wilkins Center for Study of Language and Expertise Stanford University David Fried Department of Computer Science University of Illinois at U-C November 5, 2005

lot
Download Presentation

David C. Wilkins Center for Study of Language and Expertise Stanford University David Fried

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multi-agent architectures that facilitate apprenticeship learning for real-time decision making: Minerva and Gerona David C. Wilkins Center for Study of Language and Expertise Stanford University David Fried Department of Computer Science University of Illinois at U-C November 5, 2005 Supported by ONR: N00014-00-1-0660, N00014-02-1-0731

  2. Outline • Goal • Expert shells  multi-agent capabilities • Minerva – medical diagnosis (1992-1994) • Apprentice program observes expert, improves agent • Genona – ship damage control (2002-2005) • Apprentice program observes student, improves student • Summary and conclusions

  3. Expert Shells -> Multi-Agent Capabilities • Traditional performance capabilities • Correct solution, Efficient problem solving • Multi-agent capabilities • Critiquing • Expert agent watches – finds errors omission/commission • Apprenticeship Learning • Expert agent watches expert, improves expert agent • Expert agent watches student, improves student • Research philosophy • Critiquing & apprenticeship should be natural artifact of shell architecture • Same apprenticeship method should support both learning and tutoring • Unified arch for dimensions of expertise is approach to cognitive modeling

  4. Apprenticeship Learning Paradigm Problem Human Expert Problem Solver Agent Actions Learning Actions Program KN Differences • Situated Learning: within context of problem solving • Good for knowledge refinement of human or expert agent

  5. Apprenticeship Learning Challenges • Global credit assignment • Does good explanation of human action exist? • Challenge: some explanation usually exists • Local credit assignment • What KN difference creates good explanation? • Challenge: Many repairs will create explanation • Variance among human problem solvers • How to distinguish between allowable variations among human problem solvers (who among other things often disagree) and variations that suggest knowledge errors • Solution • Minerva shell architecture

  6. Minerva-Based Apprenticeship Learning: Domain of Neurology Diagnosis 1. Debra Arbed, a 39 year old black female. 2. Chief complaint is headache, nausea, vomiting, stiff neck. 3. Headache duration? 6 hours. 4. Headache severity? 4 on scale of 0-4. 5. Fever? No. 6. Recent seizures? No. 7. Visual problems? No. 8. Headache onset? Abrupt. 30. Final diagnosis is subarachnoid hemorrhage. 31. Secondary dx is acute bacterial meningitis.

  7. Evolution of Decision-Making Expert Shells:Separation of Different Knowledge Types Minerva (1992) Odysseus2 (1994) Neomycin (1982) Guidon2 (1987) Odysseus (1988) Inference Mycin (1972) Guidon Tieresias (1978) Inference Sched Kn Task Kn Inference Task Kn Program Domain Kn Domain Kn Domain Kn

  8. Domain, Task, and Scheduling KN are Distinct • Domain KN: vocabulary and predicates mention domain • Task KN: no mention of domain (e.g., medicine): strategy(differentiate-hypotheses(Hyp1, Hyp2) :- active-hypothesis(Hyp1), active-hypothesis(Hyp1), different(Hyp1, Hyp2), evidence-for(Finding1, Hyp1, Rule1, Cf1), evidence-for(Finding1, Hyp2, Rule2, Cf2), same-sign-cfs(Cf1, Cf2), get-premise(Rule1, Finding, Premise1), get-premise( Rule2, Finding, Premiise2), premises-contradicting(Premise1, Premise2), not rule-applied(Rule1), strategy (apply-rule (Rule1)) • Scheduling KN: Chains (GSG…A) created by unification. But which Action A is best?

  9. Recursive Classification: Use in Scheduler Inference Level (Scheduler BBoard) Inference Level (Domain BBoard) Scheduler Level (Recursive HC) Scheduler Level (FIFO) Strategy Level (Exhaustive-Chaining) Strategy Level (Hypothesis-Directed) Domain Level (Scheduling knowledge) Domain Level (Medical knowledge) Minerva-Scheduler Minerva-Medicine

  10. Recursive Classification:Induction of Embedded Knowledge Base of Scheduler Rules • Induction of Scheduling rules: • 10-70 (39 avg.) classes, 42 features • 286 scheduling rules • Disjoint training and validation sets. • Critiquing evaluation • Expert’s action upper 10% = 52.2% • Expert’s action upper 20% = 67.4% • Expert’s action upper 50% = 84.8%

  11. Minerva: Related Research • Blackboard Architectures (BB1, Hearsay III) • Opaque code or scheduler hardwired: not learnable. • Classification Shells (Mole, Neomycin, Protos, Internist) • Scheduler is mostly hard-wired. • Advanced Classification Shells (Ask/Mu) • scheduler knowledge specialized 1 expert. • Critiquing Systems (Disciple, Oncocin/Protégé) • Classification vs. task reduction vs. therapy plans

  12. The Problem of Ship Damage Control • Ship crises • Fire, smoke, flooding, pipe rupture • Primary and secondary damage • Damage Control Assistant (DCA) • Responsible for overall crisis management • Makes damage control decisions • Coordinates investigation and repair teams

  13. “Damage Control Assistant” ExpertiseHow to get decision-making practice?! • Expertise requires practice • Time-critical decision-making • High stress, information overload • Uncertain and incomplete information • “Whole task” practice difficult to acquire • Actual ship crises infrequent • Realistic practice expensive and dangerous • Rotation cycle is 2-3 years

  14. The DCA Decision-Making Task:Fires, Smoke, Floods, Ruptures, etc • Event to DCA: fire observed in compartment 1-174-0-L • Event to DCA: pipe rupture observed compart 1-191-0-Q • Action by DCA: send repair party to compart 1-174-0-L • Action by DCA: go to General Quarters (GQ) • Action by DCA: start fire pump #3 on port side • Critique to DCA: Error of omission: must request permission of CO to turn on fire pump during GQ • Action by DCA: Close firemain valve 3-274-2 • Critique to DCA: Error of commission: valve 3-274-2 does not isolate pipe rupture

  15. DC-Train 4.0 Simulation Capabilities • Physical ship simulation • Primary and secondary damage • Fire, smoke, flooding, rupture, firemain • Intelligent agent personnel simulation • 67 ship personnel • Commanding officer • Engineering Officer of the Watch • Investigator Teams, Repair Teams, etc.

  16. DC-Train and SCoT-DC:Post-Scenario Spoken Dialogue Tutoring Spoken Dialogue Interface + Interactive Visualization Interface DCA student solves problem presentedby DC-Train Simulator Correct Expert Solution + Critique of Student Actions Expert & Critiquing Modules Tutoring & Dialogue Modules University of Illinois | Stanford University DC-Train 4.0 w/ Critiquing | Spoken Dialogue Tutoring

  17. Whole-Task Simulation-Based Training of Crisis Decision Making Skills Expert, Critiquing, ExplanationModels: Graph Mod Operators (GMOs, Meta-GMOs) Causal Story Graph (CSG) DC-Train: Physical Simulator and Intelligent Agents Events WorldState WorldInfo Actions Text-Based and Spoken Dialogue Tutors Event Comm Language (ECL) is used along all arrows DCA Student

  18. Gerona Expert Agent Overview • Goal: • Agent architecture to support multiple uses: • expert model, critiquing, question-answering, explanations, spoken dialogue tutoring, etc. • Solution • Explicit Knowledge Representation • ECL (vocabulary), • GMOs, G-Clauses (expert and student critique models) • Meta-GMOs (question-answering, explanations) • CSGs (structured ECLs that represent all models) • Good for knowledge acquisition from experts • Gerona representation can be “executed” by an interpreter

  19. Event Communication Language (ECL) • Event Communication Language (ECL) statements encode communication to and from the DCA, and communication about state of world. • Example • English: Boundaries set: RL5 Talker to DCA: “DCA, Repair 5 reports fire boundaries set for compartment 4-220-0-E, auxiliary machinery room #2. • ECL message 6310: Boundaries set ECL-6310 ([to], [from], “reports”, [problem], “boundaries set for compartment”, [compartment])

  20. Event Communication Language (ECL) • ECL 2000 – WorldInfo (81) • E.g., Contents of compartments, location of bulkheads • ECL 3000 –WorldState Predicates (29) • E.g., Boundaries contain compartment • ECL 4000 – WorldState Functions (22) • E.g., Compartment to Jurisdiction • ECL 5000 – Actions from the DCA (48) • E.g., Send firefighters, Start fire pump, Request permiss • ECL 6000 – Events reported to DCA (88) • E.g., Fire alarm, firemain pressure low, desmoking space • ECL 7000 – Goals (36) • E.g., Identify fire, contain fire, patch pipe rupture, • ECL 8000 – Crises (7) • E.g, Fire, hot mags, flood, smoke, pipe rupture, low fp

  21. Causal Story Graph (CSG) Crisis: Fire Active Goal: Control Fire Active Goal: Extinguish Fire Error of Commission: Fight Fire in Space Satisfied Goal: Identify Fire Active Goal: Apply Fire Suppressant Addressed Goal: Contain Fire Active Goal: Isolate Space Event: Set Fire Boundaries in progress Error of Omission: Electrically Isolate Space Justification: Why Error of Commission? Event: Fire Report Correct Action: Set Fire Boundaries

  22. Graph Modification Operators (GMO) GMO 5120 FOR ECL 5120 “Fight Fire” compartment -> Compartment target -> Station RULE 5120.fight-fire.critique.1 IF goal(find, unaddressed, 7118, “Apply fire suppressant”, [compartment = Compartment], _, G) AND action(find, pending, 5120, “Fight fire in space”, [compartment = Compartment], _, A) AND goal(find, satisfied, 7116, “Isolate compartment if necessary”, [compartment = Compartment], _, _), AND goal(find, satisfied, 7117, “Active desmoke if necessary”, [compartment = Compartment], _, _), AND ship-state(find, _, 4302, “Best repair locker for compartment”, [compartment = Compartment, station = Station], _, _)

  23. Graph Modification Operators (cont) THEN action(modify, correct, 5120, “Fight fire in space”, [compartment <- Compartment, station <- Station], _, A) goal(modify, addressed, 7118, “Apply fire suppressant”, [compartment <- Compartment], _, G) END RULE … END GMO

  24. Meta-GMO Question Types • About 100 templates cover all past instructor-student QAs • “Why” questions for justifying CSG nodes (12) • “Why should I have ordered firefighting?” • “What” questions for retrieving expert recommendations (32) • “What should I have done after I got the fire report?” • “What if” questions to get critiques on hypothetical actions (4) • “What if I ordered fire boundaries to be set?” • “When/How” questions to explain domain rules (9) • “How do you determine what repair locker has jurisdiction?” • “When/What/Is” questions evaluate conditions and relations (26) • “Is there a starboard fire pump on at 3:00?” • More complex questions involving chaining and inference (14) • “How can I satisfy the preconditions for dewatering?” • “If I ordered smoke boundaries, what could I do then?”

  25. Meta-GMO Example • “When is it appropriate to order firefighting?” • Question ECL 9300 “when action” MGMO 9300 FOR ECL 9300 “When Action” LET action-ecl-number -> ActionECL IF g-clause(find, action(create, pending, ActionECL, _, _, _, _), GClauses) g-clause(justify, GClauses, Justifications) THEN answer(create, _, 9300, “When Action”, [action-ecl-number <- ActionECL, justification <- Justification], miscellaneous-questions, JustificationNode) END IF END MGMO

  26. In English(direct translation) “There are two conditions under which you should order firefighting. “First, when you receive a report that electrical and mechanical isolation has completed, you still need to extinguish the fire in that compartment, you have either active desmoked the compartment or do not need to active desmoke the compartment, and either there is no halon or halon has failed, find the best repair locker for that compartment, and order that repair locker to fight the fire in the compartment. “Second, when you receive a report that halon has failed, you have either isolated the compartment or the compartment cannot be isolated, and you have either active desmoked the compartment or do not need to active desmoke the compartment, find the best repair locker for that compartment, and order that repair locker to fight the fire in the compartment.”

  27. In English(intelligent translation) “There are two things that might trigger ordering firefighting. The first is a report of electrical and mechanical isolation achieved, and the second is a report that halon has failed. “The first case only applies when you need to extinguish a fire. You also need to have active desmoked the compartment, if necessary, and if the compartment has halon, it has to already have failed. “In the second case, you must have active desmoked if necessary and isolated the compartment if possible. “In both cases, you should send the best repair locker for the compartment to fight the fire.”

  28. Meta-Graph Modification Operators (M-GMOs) MGMO 9002 FOR ECL 9002 "Why Sub-Optimal Action?" LET action-node -> ActionNode RULE 9002.1 "Explain why the action isn't correct." IF g-clause( find, action([create, modify], correct, ActionNode.ecl, _, _, _, _), _, CorrectGClauses) AND roll-back(before, ActionNode, _) AND g-clause(justify-and-evaluate, CorrectGClauses, ActionNode, Justification) THEN answer(create, _, 9002, "Why Sub-Optimal Action?", [action-node <- ActionNode, justification <- Justification], ActionNode, A) END RULE END MGMO

  29. Power and Learnability • A Gerona system responding to an incoming message from an agent can do so using an efficiently parallelizable algorithm. • Total space complexity is O(n) and time complexity is low-order polynomial. • GMO rules are PAC-learnable using “learning to take actions” paradigm, given certain constraints on length.

  30. Current Research Direction • Extend SCoT-DC/DC-Train Spoken Tutor to allow user-initiated tutoring. • Approach is to map user-initiated questions in natural language to Gerona question classes • QABLE for Story Comprehension Q/A (Grois and Wilkins, IJCAI-05 and ICML-05) • Use Gerona domain model to constrain interpretations (Fried, et al, 2003)

  31. Summary • Ability to critique and learn is facilitated by agent KR&I • KN factorization, explicitness, modularity, being able to reason over static and dynamic knowledge • Two examples: • Minerva: separation of domain, task, and scheduling knowledge; use of Recursive Heuristic Classification for scheduling. • Gerona: graph operators construct a dynamic task-centered representation

More Related