Lectures 5,6

1. Lectures 5,6 MACHINE LEARNING EXPERT SYSTEMS

2. Contents • Machine learning • Knowledge representation • Expert systems

3. INDUCTION OF DECISION TREES FROM DATA

4. Sport=No Sport=Yes Decision trees Outlook Sunny Rain Overcast Humidity Wind High Normal Strong Weak

5. Data from credit history of loan applications

7. A simplified tree… But how to do it?

8. The induction algorithm ID3

9. Partially constructed decision trees STEP 1 STEP 2

10. A heuristic problem HOW TO SELECT THE BEST PROPERTY?

11. Approximate trees Humidity High Normal 85% Outlook Not Sunny Sunny 100% 75%

12. CLASSIFICATION SYSTEMS

13. A full classification system

14. Pattern recognition • Patterns: • images, personal records, driving habits, etc. • Representation: • vector of features (inputs to a neural network) • Pattern classification: • Classify a pattern to one of the given classes

15. Classifier training > classifier < not Marks > classifier < Marks > classifier < not Marks > classifier < Marks > classifier < not Marks > classifier < not Marks

16. Classifier application > Classifier > Marks Note: The test image does not appear in the training data

17. LEARNING IN GENERAL

18. The data and the goals • We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt) • The goal is to infer a general definition that will allow the learner to recognize future instances of the class

19. Knowledge representation • Positive and negative examples can be represented, e.g., in predicate calculus • Two positive instances of the concept of “ball” can be expressed as follows: size(obj1,small)  color(obj1,red) shape(obj1,round) size(obj2,large)  color(obj2,red) shape(obj2,round) • The general concept of “ball” could be defined by: size(X,Y)  color(X,Z) shape(X,round) where any sentence that unifies with this general definition represents a ball

20. A general model of the learning process

21. A set of operations Given a set of training instances, the learner must construct a generalization, heuristic rule or plan that satisfies its goals

22. The concept space Representation language and the operations define a space of potential concept definitions The learner must search this space to find the desired concept

23. Heuristic search Learning programs must commit to a direction and order of search, as well as… …to the use of available data and heuristics to search efficiently

24. PATRICK WINSTON’S PROGRAM ON LEARNING CONCEPTS

25. Examples and near misses for the concept “arch”

26. Generalization of descriptions to include multiple examples (I)

27. Generalization of descriptions to include multiple examples (II)

28. Starting with the original Specialization of a description to exclude a near miss we add special constraints so that this can’t match

30. A BRIEF HISTORY OF AI REPRESENTATIONAL SCHEMES

31. developed by Collins & Quillian in their research on human information storage and response times Semantic network

32. Network representation of properties of… …snow and ice

33. Three planes representing… …three definitions of the word “plant”

34. Intersection path between “cry” and “comfort” (Quillian 1967)

35. Case frame representation of the sentence “Sarah fixed thechair with glue.”

36. Conceptual dependency theory of four primitive conceptualizations For example, all actions are assumed to reduce to one or more of the primitive ACTs listed below:

38. “John ate the egg” “John prevented Mary from giving a book to Bill”

39. Restaurant script (Schank and Abelson 1977)

40. Restaurant script (continued)

41. FRAMES

42. A frame includes: • Frame identification information • Its relationship to other frames • Descriptors of requirements • Procedural information on use of the structure described • Frame default information • New instance information

43. Relationship to other frames • For instance, the “hotel phone” might be a special instance of “phone”, which might be an instance of a “communication device”

44. Descriptors of requirements • For instance, a chair has its seat between 20 and 40 cm from the floor, its back higher than 60 cm, etc. • These requirements may be used to determine when new objects fir the stereotype defined by the frame

45. Procedural information • An important feature of frames is the ability to attach procedural code to a slot

46. Frame default information • These are slot values that are taken to be true when no evidence to the contrary has been found • For instance, chairs have four legs, telephones are pushbutton, hotel beds are made by the staff

47. New instance information • Many frame slots may be left unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving • For instance, the color of the bedspread may be left unspecified

48. Part of a frame description of a hotel room “Specialization” indicates a pointer to a superclass

49. Spatial frame for viewing a cube (Minsky 1975)

50. CONCEPTUAL GRAPHS: A NETWORK LANGUAGE