Learning Agents Laboratory Computer Science Department George Mason University

1. CS 782 Machine Learning 10 Multistrategy Learning Prof. Gheorghe Tecuci Learning Agents Laboratory Computer Science Department George Mason University

2. Overview Introduction Combining EBL with Version Spaces Induction over Unexplained Guiding Induction by Domain Theory Plausible Justification Trees Research Issues Basic references

3. Multistrategy learning Multistrategy learning is concerned with developing learning agents that synergistically integrate two or more learning strategies in order to solve learning tasks that are beyond the capabilities of the individual learning strategies that are integrated.

4. Complementariness of learning strategies several many one complete incomplete very little knowledge knowledge induction and/ deduction induction or deduction improves improves improves competence or/ competence efficiency and efficiency Case Study: Inductive Learning vs Explanation-based Learning Explanation- based learning Multistrategy Learningfrom examples learning Examples needed Knowledge needed Type of inference Effect on agent's behavior

5. Multistrategy concept learning The Learning Problem Input Background Knowledge (Domain Theory) Goal One or more positive and/or negative examples of a concept Weak, incomplete, partially incorrect, or complete Learn a concept description characterizing the example(s) and consistent with the background knowledge by combining several learning strategies

6. Multistrategy knowledge base refinement The Learning Problem: Improve the knowledge base so that the Inference Engine solves (classifies) correctly the training examples. Training Examples Improved Knowledge Multistrategy Base (DT) Knowledge Knowledge Base Base (DT) Refinement Inference Engine Inference Engine Similar names: background knowledge – domain theory – knowledge base knowledge base refinement - theory revision

7. Types of theory errors (in a rule based system) has-handle(x) shape(x, round) insulating(x) width(x, small)& insulating(x)& graspable(x) graspable(x) graspable(x) shape(x, round) graspable(x) How would you call a KB where some positive examples are not explained (classified as positive)? How would you call a KB where some negative examples are wrongly explained (classified as positive)? Incorrect KB (theory) + + + _ Overly Overly Specific General Missing Extra Additional Missing Rule Rule Premise Premise width(x, small) What is the effect of each error on the system’s ability to classify graspable objects, or other objects that need to be graspable, such as cups? Positive examples Negative examples Proofs for some positive examples cannot be built: Proofs for some negative examples can be built: Positive examples that are not round, or have a handle Negative examples that are round, or are insulating but not small

8. Overview Introduction Combining EBL with Version Spaces Induction over Unexplained Guiding Induction by Domain Theory Plausible Justification Trees Research Issues Basic references

9. EBL-VS: Combining EBL with Version Spaces • Apply explanation-based learning to generalize the positive and the negative examples. • Replace each example that has been generalized with its generalization. • Apply the version space method to the new set of examples. Produce an abstract illustration of this algorithm.

10. EBL-VS features • Apply explanation-based learning to generalize the positive and the negative examples. • Replace each example that has been generalized with its generalization. • Apply the version space method to the new set of examples. Justify the following feature, considering several cases: • Learns from positive and negative examples

11. EBL-VS features • Apply explanation-based learning to generalize the positive and the negative examples. • Replace each example that has been generalized with its generalization. • Apply the version space method to the new set of examples. Justify the following feature: • Can learn with an incomplete background knowledge

12. EBL-VS features • Apply explanation-based learning to generalize the positive and the negative examples. • Replace each example that has been generalized with its generalization. • Apply the version space method to the new set of examples. Justify the following feature: • Can learn with different amounts of knowledge, from knowledge-free to knowledge-rich

13. EBL-VS features summary and references • Learns from positive and negative examples • Can learn with an incomplete background knowledge • Can learn with different amounts of knowledge, from knowledge-free to knowledge-rich References Hirsh, H., "Combining Empirical and Analytical Learning with Version Spaces," in Proc. of the Sixth International Workshop on Machine Learning, A. M. Segre (Ed.), Cornell University, Ithaca, New York, June 26-27, 1989. Hirsh, H., "Incremental Version-space Merging," in Proceedings of the 7th International Machine Learning Conference, B.W. Porter and R.J. Mooney (Eds.), Austin, TX, 1990.

14. Overview Introduction Combining EBL with Version Spaces Induction over Unexplained Guiding Induction by Domain Theory Plausible Justification Trees Research Issues Basic references

15. IOU: Induction Over Unexplained Justify the following limitation of EBL-VS: Limitation of EBL-VS • Assumes that at least one generalization of an example is correct and complete IOU • Knowledge base could be incomplete but correct: - the explanation-based generalization of an example may be incomplete; - the knowledge base may explain negative examples. • Learns concepts with both explainable and conventional aspects

16. IOU method • Apply explanation-based learning to generalize each positive example • Disjunctively combine these generalizations (this is the explanatory component Ce) • Disregard negative examples not satisfying Ce and remove the features mentioned in Ce from all the examples • Apply empirical inductive learning to determine a generalization of the reduced set of simplified examples (this is the non-explanatory component Cn) The learned concept is Ce & Cn

17. IOU: illustration Positive examples of cups: Cup1, Cup2 Negative examples: Shot-Glass1, Mug1, Can1 Domain Theory: incomplete - contains a definition of a generalization of the concept to be learned (e.g. contains a definition of drinking vessels but no definition of cups) Ce = has-flat-bottom(x) & light(x) & up-concave(x) & {[width(x,small) & insulating(x)]has-handle(x)} Ce covers Cup1, Cup2, Shot-Glass1, Mug1 but not Can1 Cn = volume(x,small) Cn covers Cup1, Cup2 but not Shot-Glass1, Mug1 C = Ce & Cn Mooney, R.J. and Ourston, D., "Induction Over Unexplained: Integrated Learning of Concepts with Both Explainable and Conventional Aspects,", in Proc. of the Sixth International Workshop on Machine Learning, A.M. Segre (Ed.), Cornell University, Ithaca, New York, June 26-27, 1989.

18. Overview Introduction Combining EBL with Version Spaces Induction over Unexplained Guiding Induction by Domain Theory Plausible Justification Trees Research Issues Basic references

19. Enigma: Guiding Induction by Domain Theory Justify the following limitations of IOU: Limitations of IOU • Knowledge base rules have to be correct • Examples have to be noise-free ENIGMA • Knowledge base rules could be partially incorrect • Examples may be noisy

20. Enigma: method Trades-off the use of knowledge base rules against the coverage of examples: • Successively specialize the abstract definition D of the concept to be learned by applying KB rules • Whenever a specialization of the definition D contains operational predicates, compare it with the examples to identify the covered and the uncovered ones • Decide between performing: - a KB-based deductive specialization of D - an example-based inductive modification of D The learned concept is a disjunction of leaves of the specialization tree built.

21. Enigma: illustration Examples (4 positive, 4 negative) Positive example4 (p4): light(o4) & support(o4,b) & body(o4,a) & above(a,b) & up-concave(o4)  Cup(o4) Background Knowledge Liftable(x) & Stable(x) & Open-vessel(x)  Cup(x) light(x) & has-handle(x)  Liftable(x) has-flat-bottom(x)  Stable(x) body(x, y) & support(x, z) & above(y, z)  Stable(x) up-concave(x)  Open-vessel(x) KB: - partly overly specific (explains only p1 and p2) - partly overly general (explains n3) Operational predicates start with a lower-case letter

22. Enigma: illustration (cont.) Classification is based only on operational features: (to cover p3,p4) (to uncover n2,n3)

23. Learned concept light(x) & has-flat-bottom(x) &has-small-bottom(x)  Cup(x) Covers p1, p3 light(x) & body(x, y) & support(x, z) & above(y, z) & up-concave(x)  Cup(x) Covers p2, p4

24. Application • Diagnosis of faults in electro-mechanical devices through an analysis of their vibrations • 209 examples and 6 classes • Typical example: 20 to 60 noisy measurements taken in different points and conditions of the device • A learned rule: IF the shaft rotating frequency is w0 and the harmonic at w0 has high intensity and the harmonic at 2w0 has high intensity in at least two measurements THEN the example is an instance of C1 (problems in the joint), C4 (basement distortion) or C5 (unbalance)

25. Application (cont.) Comparison betweenthe KB learned by ENIGMA and the hand-coded KB of the expert system MEPS Bergadano, F., Giordana, A. and Saitta, L., "Automated Concept Acquisition in Noisy Environments," IEEE Transactions on Pattern Analysis and Machine Intelligence, 10 (4), pp. 555-577, 1988. Bergadano, F., Giordana, A., Saitta, L., De Marchi D. and Brancadori, F., "Integrated Learning in a Real Domain," in B.W. Porter and R.J. Mooney (Eds. ), Proceedings of the 7th International Machine Learning Conference, Austin, TX, 1990. Bergadano, F. and Giordana, A., "Guiding Induction with Domain Theories," in Machine Learning: An Artificial Intelligence Approach Vollume 3, Y. Kodratoff and R.S. Michalski (Eds.), San Mateo, CA, Morgan Kaufmann, 1990.

26. Overview Introduction Combining EBL with Version Spaces Induction over Unexplained Guiding Induction by Domain Theory Plausible Justification Trees Research Issues Basic references

27. MTL-JT: Multistrategy Task-adaptive Learning based on Plausible Justification Trees • Deep integration of learning strategies Integration of the elementary inferences that are employed by the single-strategy learning methods (e.g. deduction, analogy, empirical inductive prediction, abduction, deductive generalization, inductive generalization, inductive specialization, analogy-based generalization). • Dynamic integration of learning strategies The order and the type of the integrated strategies depend of the relationship between the input information, the background knowledge and the learning goal. • Different types of input (e.g. facts, concept examples, problem solving episodes) • Different types of knowledge pieces in the knowledge base (e.g. facts, examples, implicative relationships, plausible determinations)

28. MTL-JT: assumptions Input: • correct (noise free) • one or several examples, facts, or problem solving episodes Knowledge Base: • incomplete and/or partially incorrect • may include a variety of knowledge types (facts, examples, implicative or causal relationships, hierarchies, etc.) Learning Goal: • extend, update and/or improve the knowledge base so as to integrate new input information

29. Plausible justification tree A plausible justification tree is like a proof tree, except that some of individual inference steps are deductive, while others are non-deductive or only plausible (e.g. analogical, abductive, inductive).

30. Learning method • For the first positive example I1: - build a plausible justification tree T of I1 - build the plausible generalization Tu of T - generalize the KB to entail Tu • For each new positive example Ii: - generalize Tu so as to cover a plausible justification tree of Ii - generalize the KB to entail the new Tu • For each new negative example Ii: - specialize Tu so as not to cover any plausible justification of Ii - specialize the KB to entail the new Tu without entailing the previous Tu • Learn different concept definitions: - extract different concept definitions from the general justification tree Tu

31. MTL-JT: illustration from Geography Knowledge Base Facts: terrain(Philippines, flat), rainfall(Philippines, heavy), water-in-soil(Philippines, high) Examples (of fertile soil): soil(Greece, red-soil)  soil(Greece, fertile-soil) terrain(Egypt, flat) & soil(Egypt, red-soil)  soil(Egypt, fertile-soil) Plausible determination: rainfall(x, y) >= water-in-soil(x, z) Deductive rules: soil(x, loamy)  soil(x, fertile-soil) climate(x, subtropical)  temperature(x, warm) climate(x, tropical)  temperature(x, warm) water-in-soil(x, high) & temperature(x, warm) & soil(x, fertile-soil)  grows(x, rice)

32. Positive and negative examples of "grows(x, rice)" Positive Example 1: rainfall(Thailand, heavy) & climate(Thailand, tropical) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia)  grows(Thailand, rice) Positive Example 2: rainfall(Pakistan, heavy) & climate(Pakistan, subtropical) & soil(Pakistan, loamy) & terrain(Pakistan, flat) & location(Pakistan, SW-Asia)  grows(Pakistan, rice) Negative Example 3: rainfall(Jamaica, heavy) & climate(Jamaica, tropical) & soil(Jamaica, loamy) & terrain(Jamaica, abrupt) & location(Jamaica, Central-America)  ¬ grows(Jamaica, rice)

33. Build a plausible justification of the first example Example 1: rainfall(Thailand, heavy) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia) & climate(Thailand, tropical) grows(Thailand, rice)

34. Build a plausible justification of the first example Example 1: rainfall(Thailand, heavy) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia) & climate(Thailand, tropical)  grows(Thailand, rice) Justify the inferences from the above tree: analogy Facts: terrain(Philippines, flat), rainfall(Philippines, heavy), water-in-soil(Philippines, high) Plausible determination: rainfall(x, y) >= water-in-soil(x, z)

35. Build a plausible justification of the first example Example 1: rainfall(Thailand, heavy) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia) & climate(Thailand, tropical)  grows(Thailand, rice) Justify the inferences from the above tree: deduction Deductive rules: soil(x, loamy)  soil(x, fertile-soil) climate(x, subtropical)  temperature(x, warm) climate(x, tropical)  temperature(x, warm) water-in-soil(x, high) & temperature(x, warm) & soil(x, fertile-soil)  grows(x, rice)

36. Build a plausible justification of the first example Example 1: rainfall(Thailand, heavy) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia) & climate(Thailand, tropical)  grows(Thailand, rice) Justify the inferences from the above tree: inductive prediction & abduction Examples (of fertile soil): soil(Greece, red-soil)  soil(Greece, fertile-soil) terrain(Egypt, flat) & soil(Egypt, red-soil)  soil(Egypt, fertile-soil)

37. Multitype generalization

38. Multitype generalization Justify the generalizations from the above tree: generalization based on analogy Facts: terrain(Philippines, flat), rainfall(Philippines, heavy), water-in-soil(Philippines, high) Plausible determination: rainfall(x, y) >= water-in-soil(x, z)

39. Multitype generalization Justify the generalizations from the above tree: Inductive generalization Examples (of fertile soil): soil(Greece, red-soil)  soil(Greece, fertile-soil) terrain(Egypt, flat) & soil(Egypt, red-soil)  soil(Egypt, fertile-soil)

40. Build the plausible generalization Tu of T

41. Positive example 2 Instance of the current Tu corresponding to Example 2 Plausible justification tree T2 of Example 2:

42. Positive example 2 The explanation structure S2: The new Tu:

43. Negative example 3 Instance of Tu corresponding to the Negative Example 3: The new Tu:

44. The plausible generalization tree corresponding to the three input examples

45. Learned knowledge New facts: water-in-soil(Thailand, high) water-in-soil(Pakistan, high) Why is it reasonable to consider these facts to be true?

46. Learned knowledge New plausible rule: soil(x, red-soil)  soil(x, fertile-soil) Examples (of fertile soil): soil(Greece, red-soil)  soil(Greece, fertile-soil) terrain(Egypt, flat) & soil(Egypt, red-soil)  soil(Egypt, fertile-soil)

47. Learned knowledge Specialized plausible determination: rainfall(x, y) & terrain(x, flat) >= water-in-soil(x, z) Facts: terrain(Philippines, flat), rainfall(Philippines, heavy), water-in-soil(Philippines, high) Positive Example 1: rainfall(Thailand, heavy) & climate(Thailand, tropical) & soil(Thailand, red-soil) & terrain(Thailand, flat) & location(Thailand, SE-Asia)  grows(Thailand, rice) Positive Example 2: rainfall(Pakistan, heavy) & climate(Pakistan, subtropical) & soil(Pakistan, loamy) & terrain(Pakistan, flat) & location(Pakistan, SW-Asia)  grows(Pakistan, rice) Negative Example 3: rainfall(Jamaica, heavy) & climate(Jamaica, tropical) & soil(Jamaica, loamy) & terrain(Jamaica, abrupt) & location(Jamaica, Central-America)  ¬ grows(Jamaica, rice)

48. Learned knowledge: concept definitions Operational definition of "grows(x, rice)": rainfall(x,heavy) & terrain(x,flat) & [climate(x,tropical)  climate(x,subtropical)] & [soil(x,red-soil)  soil(x,loamy)]  grows(x, rice) Abstract definition of "grows(x, rice)": water-in-soil(x, high) & temperature(x, warm) & soil(x, fertile-soil)  grows(x, rice)

49. Learned knowledge: example abstraction Abstraction of Example 1: water-in-soil(Thailand, high) & temperature(Thailand, warm) & soil(Thailand, fertile-soil)  grows(Thailand, rice)

50. Features of the MTL-JT method and reference • Is general and extensible • Integrates dynamically different elementary inferences • Uses different types of generalizations • Is able to learn from different types of input • Is able to learn different types of knowledge • Exhibits synergistic behavior • May behave as any of the integrated strategies Tecuci, G., "An Inference-Based Framework for Multistrategy Learning," in Machine Learning: A Multistrategy Approach Volume 4, R.S. Michalski and G. Tecuci (Eds.), San Mateo, CA, Morgan Kaufmann, 1994.