1 / 53

Knowledge Systems and Project Halo

Knowledge Systems and Project Halo. In collaboration with SRI (Vinay Chaudhri) and Boeing (Peter Clark). Knowledge Systems. Knowledge Systems are formal representations of knowledge capable of answering unanticipated questions with coherent explanations

hamilton-klein
Download Presentation

Knowledge Systems and Project Halo

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Knowledge Systems and Project Halo In collaboration with SRI (Vinay Chaudhri) and Boeing (Peter Clark)

  2. Knowledge Systems • Knowledge Systems are formal representations of knowledge capable of answering unanticipated questions with coherent explanations • Knowledge System = KB + Q/A + Explanation Generator + Knowledge Acq. tools

  3. Project Halo • Funded and administered by Vulcan, Inc – a Paul Allen company • Objective: to assess the state of the art of knowledge systems – computer programs that know a lot and answer tough questions with coherent explanations • Method: administer an AP Chemistry exam to knowledge systems built by 4 teams of researchers

  4. A Significant Advance over Expert Systems • Coverage • Reasoning • Explanation • Rapid construction

  5. KM: A Logic Programming Language • …able to represent: • classes, instances, prototypes • defaults, fluents, constraints • (hypothetical) situations • actions (pre-, post-, and during- conditions) • …and reason about: • inheritance with exceptions • deductive and abductive inference (with constraints) • automatic classification (given a partial description of an instance, determine the classes to which it belongs) • temporal projection (“my car is where I left it”) • affects of actions

  6. A Simple Example • When 70 ml of 3.0-Molar Na2CO3 is added to 30 ml of 1.0-Molar NaHCO3 the resulting concentration of Na+ is: • 2.0 M • 2.4 M • 4.0 M • 4.5 M • 7.0 M

  7. output Question 26 context result Mix Mixture has-part raw material Aqueous Solution Aqueous Solution Na+ base conc. volume conc. base volume conc. ?? 3.0 M 0.07 lit 1.0 M 0.03 lit Na2CO3 NaHCO3 Question Representation

  8. Background Knowledge Chemistry laws: • Concentration of a solute • Composition of strong electrolyte solutions • Conservation of mass • Conservation of volume etc.

  9. Compute-Concentration Method output context input Mixture has-part volume conc. Concentration *molar Chemical Volume *liters quantity Quantity *moles Explanation Template The concentration of a chemical in a mixture is the quantity of the chemical divided by the volume of the mixture. Divide the quantity by the volume: <Quantity> / <Volume> = X *molar Therefore, the concentration of <Chemical> in <Mixture> = X *molar Law 1: Concentration of a Solute Note: when this law is applied, using Novak’s code, the quantities are automatically converted to the units-of-measurement specified here

  10. Compute-Ions-in-Strong-Electrolyte context output input Strong Electrolyte quantity has-part Quantity *moles Cation Anion quantity quantity Quantity *moles Quantity *moles Law 2: Composition of Strong Electrolytes

  11. Conservation of Mass output input context Chemical result Mix has-part raw-material part-of Chemical … Chemical1 Chemicaln quantity quantity quantity ?? *moles Quantity *moles Quantity *moles Explanation Template By the Law of Conservation of Mass, the quantity of a chemical in a mixture is the sum of the quantities of that chemical in the parts of the mix. The quantity of <Chemical> in <Chemical1> is X1 *moles … The quantity of <Chemical> in <Chemicaln> is Xn *moles Therefore, the quantity of <Chemical> = X *moles Law 3: Conservation of Mass

  12. Conservation of Volume output Mixture result input Mix context volume raw-material ?? *liter … Chemical1 Chemicaln volume volume Volume <uom1> Volume <uomn> Explanation Template By the Law of Conservation of Volume, the volume of a mixture is the sum of the volumes of the parts mixed. The sum of X1 <uom1>, … and Xn <uomn> = X *liter Therefore, the volume of <Mixture> = X *liter Law 4: Conservation of Volume

  13. Strong Electrolyte Solution result superclass Mix Mixture has-part raw material Aqueous Solution Aqueous Solution Na+ base conc. volume base volume conc. 3.0 M 0.07 lit 1.0 M 0.03 lit Na2CO3 NaHCO3 Step 1: Reclassify Terms

  14. volume ?? *liters quantity Mixture ?? *moles conc. ?? *molar has-part volume conc. Concentration *molar Chemical Volume *liters result Mix quantity Mixture has-part raw material Quantity *moles Aqueous Solution Aqueous Solution Na+ base volume conc. conc. volume base Law 1 3.0 M 0.07 lit 1.0 M 0.03 lit Na2CO3 NaHCO3 Step 2: Use Law 1 to Compute Concentration

  15. The Search is non-deterministic • Multiple laws might be used to compute a value for any property. For example, here’s another way to compute concentration: • pH = - log [H+], where [H+] is the concentration of H+ • Since this applies only to H+, this search path ends quickly

  16. volume ?? *liters .1 quantity ?? *moles conc. Chemical result Mix ?? *molar volume raw-material Volume *liter … result Chemical Chemical Mix Mixture volume has-part volume raw material Law 4 Aqueous Solution Aqueous Solution Na+ Volume *liter Volume *liter base volume conc. conc. volume base 3.0 M 0.07 lit 1.0 M 0.03 lit Na2CO3 NaHCO3 Step 3: Use Law 4 to Compute Volume

  17. volume .1 *liters Mix Mixture result raw material has-part Chemical result Mix quantity Aqueous Solution Aqueous Solution Na+ has-part base base conc. raw-material volume volume conc. conc. ?? *moles part-of Na2CO3 NaHCO3 0.03 liters ?? *molar Chemical 0.07 liters … Chemical Chemical 1.0 M 3.0 M quantity quantity quantity has-part ?? *moles Quantity *moles Quantity *moles Na+ Na+ Law 3 quantity ?? *moles ?? *moles Step 4: Use Law 3 to Compute Quantity

  18. volume .1 *liters Mix Mixture result raw material has-part quantity Aqueous Solution Aqueous Solution Strong Electrolyte Na+ base base conc. volume volume conc. conc. ?? *moles quantity Na2CO3 NaHCO3 0.03 liters ?? *molar 0.07 liters has-part Quantity *moles 1.0 M 3.0 M Cation Anion has-part Law 2 quantity Na+ Na+ quantity quantity quantity Quantity *moles Quantity *moles ?? *moles ?? *moles ?? *moles Step 5: Use Law 2 to Compute Quantity of Ionic Parts

  19. volume .1 *liters Mix Mixture result raw material has-part quantity Aqueous Solution Aqueous Solution Na+ base base conc. volume volume conc. conc. ?? *moles Na2CO3 NaHCO3 0.03 liters ?? *molar 0.07 liters 1.0 M 3.0 M Mixture has-part quantity Na+ Na+ has-part volume quantity conc. Concentration *molar ?? *moles Chemical ?? *moles ?? *moles Volume *liters quantity .21 Law 1’ Quantity *moles Step 6: Use Law 1’ to Compute Quantity

  20. volume .1 *liters Mix Mixture result raw material has-part quantity Aqueous Solution Aqueous Solution Strong Electrolyte Na+ base base conc. volume volume conc. conc. ?? *moles quantity Na2CO3 NaHCO3 0.03 liters ?? *molar 0.07 liters has-part Quantity *moles 1.0 M 3.0 M Cation Anion has-part Law 2 quantity Na+ Na+ quantity quantity quantity Quantity *moles Quantity *moles .42 .21 *moles ?? *moles ?? *moles Step 7: Wind out of Law 2 from step 5

  21. volume .1 *liters Mix Mixture result raw material has-part quantity Aqueous Solution Aqueous Solution Na+ base base conc. volume volume conc. conc. ?? *moles Na2CO3 NaHCO3 0.03 liters ?? *molar 0.07 liters 1.0 M 3.0 M has-part quantity Na+ Na+ quantity .03 .21 *moles .42 *moles ?? *moles Step 8-10: Similar to steps 5-7

  22. volume .1 *liters Mix Mixture result raw material has-part Chemical result Mix quantity Aqueous Solution Aqueous Solution Na+ .45 has-part base base conc. raw-material volume volume conc. conc. ?? *moles part-of Na2CO3 NaHCO3 0.03 liters ?? *molar Chemical 0.07 liters … Chemical Chemical 1.0 M 3.0 M quantity quantity quantity has-part ?? *moles Quantity *moles quantity Quantity *moles Na+ Na+ Law 3 quantity .21 *moles .42 *moles .03 *moles Step 11: Wind out of Law 3 from Step 4

  23. volume .1 *liters Mix Mixture result raw material has-part quantity Aqueous Solution Aqueous Solution Na+ Mixture base conc. volume volume conc. conc. .45 *moles Na2CO3 0.03 liters ?? *molar 0.07 liters base has-part volume NaHCO3 4.5 1.0 M 3.0 M conc. Concentration *molar Chemical has-part Volume *liters quantity quantity Na+ Na+ Quantity *moles quantity .21 *moles .42 *moles .03 *moles Law 1 Step 12: Wind out of Law 1 from Step 2

  24. Question 26 Answer When 70 ml of 3.0-Molar Na2CO3 is added to 30 ml of 1.0-Molar NaHCO3, what is the resulting concentration of Na+?. The concentration of a chemical in a mixture is the quantity of the chemical divided by the volume of the mixture. By the Law of Conservation of Mass, the quantity of a chemical in a mixture is the sum of the quantities of that chemical in the parts of the mix. In the na2co3 strong-electrolyte-solution and the nahco3 strong-electrolyte-solution : In the na-plus : Multiply the concentration and the volume: 3 molar * 70 milliliter = 0.21 mole. The quantity of na-plus in the na-plus is 0.42 mole. In the co3-2 : The quantity of na-plus in the co3-2 is 0 mole. Multiply the concentration and the volume: 1 molar * 30 milliliter = 0.03 mole. In the na-plus : The quantity of na-plus in the na-plus is 0.03 mole. In the hco3- : The quantity of na-plus in the hco3- is 0 mole. The quantity of na-plus in the na2co3 strong-electrolyte-solution and the nahco3 strong-electrolyte-solution is 0.45 mole. Therefore, the quantity of na-plus = 0.45 mole. By the Law of Conservation of Volume, the volume of a mixture is the sum of the volumes of the parts mixed. The sum of 70 milliliter and 30 milliliter = 0.10 liter. Therefore, the volume of the strong-electrolyte-solution strong-electrolyte-solution mixture = 0.10 liter. Divide the quantity by the volume:. 0.45 mole / 0.10 liter = 4.50 molar. Therefore, the concentration of na-plus in the strong-electrolyte-solution strong-electrolyte-solution mixture = 4.50 molar. When 70 ml of 3.0-Molar Na2CO3 is added to 30 ml of 1.0-Molar NaHCO3, the resulting concentration of Na+ is 4.50 molar

  25. Results of Project Halo • After 4 month development effort, the knowledge systems were sequestered and given a test: • 165 novel questions: 50 multiple choice; 115 free form response • Questions translated from English to formal language by each team, then assessed for fidelity by an independent committee • High likelihood of long term follow on

  26. Correctness • The SRI’s team correctness score corresponds to an AP score of 3 – high enough for credit at UCSD, UIUC, and many other universities. • We’ve predicted scoring 85% after a 3 month follow-on project.

  27. Explanation Quality

  28. Our Long Term Goal • to enable distributed communities of domain experts to build knowledge systems in their area of expertise … • without direct help from knowledge engineers • working with familiar concepts and without writing axioms • with little more effort than writing technical papers

  29. Our Current Focus • Insight: even domain-specific representations contain common abstractions • Approach: we build a library consisting of • a small hierarchy of reusable, composable, domain-independent knowledge units (“components”) • a small vocabulary of relations to connect them then domain experts build representations by instantiating and composing these components

  30. Building a Representation Compositionally Soil Rate contains I- I- Q+ environment Q- rate agent Bio- technologist Bioremediation Amount Amount amount amount script remediator product pollutant agent Microbes Script Oil Fertilizer patient se se se se patient agent absorbed product Break Down Get Apply Absorb then then then

  31. An underlying abstraction... Soil Rate contains I- I- Q+ environment Q- rate agent Bio- technologist Bioremediation Amount Amount amount amount script remediator product pollutant agent Microbes Script Oil Fertilizer patient se se se se patient agent absorbed product Break Down Get Apply Absorb then then then Rate I- I- Q+ Q- rate Amount Amount Conversion amount raw- materials amount product Substance Substance

  32. Another abstraction... Soil Rate contains I- I- Q+ environment Q- rate agent Bio- technologist Bioremediation Amount Amount amount amount script remediator product pollutant agent Microbes Script Oil Fertilizer se se se patient se patient agent absorbed product Break Down Get Apply Absorb then then then Digest food eater script Agent Script Substance agent se patient se absorbed agent Break Down Absorb then

  33. Another abstraction... Agent Soil Rate contains I- I- Q+ environment Q- rate agent Bio- technologist Bioremediation Amount Amount amount amount script remediator product pollutant agent Microbes Script Oil Fertilizer patient se se se se patient agent absorbed product Break Down Get Apply Absorb then then then Treatment script substance Script substance se patient patient Get Apply then

  34. Examples of Concepts Described Compositionally • a Fuel-Cell is a Producer of Electricity • a Bulb is an Electrical Resistor that ProducesLight • a Camera is an Image Recording Device • a Wire is a Conduit of Electricity

  35. Library Contents • actions — things that happen, change states • Enter, Copy, Replace, Transfer, etc. • states — relatively temporally stable events • Be-Closed, Be-Attached-To, Be-Confined, etc. • entities — things that are • Substance, Place, Object, etc. • roles — things that are, but only in the context of things that happen • Container, Catalyst, Barrier, Vehicle, etc.

  36. Library Contents • relations between events, entities, roles • agent, donor, object, recipient, result, etc. • content, part, material, possession, etc. • causes, defeats, enables, prevents, etc. • purpose, plays, etc. • properties between events/entities and values • rate, frequency, intensity, direction, etc. • size, color, integrity, shape, etc.

  37. Computational Semantics • Knowledge about Enter: • instances of Enter inherit axioms from Move, such as: the action changes the location of the object of the Move • before the Enter, the object is outside some enclosure • after the Enter, the object is inside that enclosure and contained by it • during the Enter, the object passes through a portal of the enclosure • if the portal has a covering, it must be open; and unless it is known to be closed, assume that it’s open • etc.

  38. Searching the Library • browsing the hierarchy top-down • WordNet-based search • all components have hooks to WordNet • climb the WordNet hypernym tree with search terms • assemble: Attach,Come-Togethermend: Repairinfiltrate: Enter,Traverse,Penetrate,Move-Intogum-up: Block, Obstructbusted: Be-Broken,Be-Ruined

  39. First Challenge Problem • To enable biologists to encode college-level textbook knowledge about cells • A small example: mRNA-Transport • “mRNA is transported out of the cell nucleus into the cytoplasm” • Transport: Move-Out-Of

  40. unify

  41. location

  42. Evaluation • Can Domain Experts learn to use the library to encode domain knowledge? • Can sophisticated knowledge be captured through composition of components?

  43. Methodology • train biologists (4 graduate students) for six days • have them encode knowledge from a college textbook, Essential Cell Biology by Bruce Alberts • supply end-of-the-chapter-style Biology questions • have the biologists pose the questions to their knowledge bases and record the answers • have another biologist evaluate the answers on a scale of 0-3 • qualitatively evaluate their KBs

  44. Some Example Questions What nucleotide base pairs with adenine in RNA? How is uracil in RNA like thymine in DNA? What is the relationship between thymine and uracil? For a given bacterial gene, how are bacterial RNA and DNA molecules different? Describe RNA as a kind of polymer. What are the four bases/nucleotides of RNA? What is the relationship between a DNA gene and its RNA transcription product?

  45. Evaluation — Productivity

More Related