1. Machine Learning A Quick look • Sources: • Artificial Intelligence – Russell & Norvig • Artifical Intelligence - Luger By: Héctor Muñoz-Avila

2. Environment Environment System Action1 Action2 Knowledge Knowledge changed What Is Machine Learning? “Logic is not the end of wisdom, it is just the beginning” --- Spock time System

3. Learning: The Big Picture • Two forms of learning: • Supervised: the input and output of the learning component can be perceived (for example: friendly teacher) • Unsupervised: there is no hint about the correct answers of the learning component (for example to find clusters of data)

4. Offline vs. Online Learning • Online – during gameplay • Adapt to player tactics • Avoid repetition of mistakes • Requirements: computationally cheap, effective, robust, fast learning (Spronck 2004) • Offline - before the game is released • Devise new tactics • Discover exploits

5. Classification(According to the language representation) • Symbolic • Version Space • Decision Trees • Explanation-Based Learning • … • Sub-symbolic • Reinforcement Learning • Connectionist • Evolutionary

6. Classification(According to the language representation) • Symbolic • Version Space • Decision Trees • Explanation-Based Learning • … • Sub-symbolic • Reinforcement Learning • Connectionist • Evolutionary

7. Two extremes (temptative) solutions: too general obj(X,Y,Z) … obj(X,Y,ball) concept space obj(large,Y,ball) obj(small,Y,ball) • obj(large,white,ball) obj(small,blue,ball) … too specific Version Space Idea: Learn a concept from a group of instances, some positive and some negative • Example: • target: obj(Size,Color,Shape) • Size = {large, small} • Color = {red, white, blue} • Shape = {ball, brick, cube} • Instances: • +: • obj(large,white,ball) • obj(small,blue,ball) • −: • obj(small,red,brick) • obj(large,blue,cube)

8. How Version Space Works If we consider positive and negatives If we consider only positives + + + + + + + + + + + + + − + − − − + + What is the role of the negative instances? to help prevent over-generalizations

9. Classification(According to the language representation) • Symbolic • Version Space • Decision Trees • Explanation-Based Learning • … • Sub-symbolic • Reinforcement Learning • Connectionist • Evolutionary

10. Explanation-Based learning C A B C A B C B A B A C ? B A ? C B ? A B C C B A B A C A C A A B C B C C B A A B C Can we avoid making this error again?

11. Possible rule: If the initial state is this and the final state is this, don’t do that Explanation-Based learning (2) C A B ? B A ? C ? A B C C B A B A C More sensible rule: don’t stack anything above a block, if the block has to be free in the final state

12. Classification(According to the language representation) • Symbolic • Version Space • Decision Trees • Explanation-Based Learning • … • Sub-symbolic • Reinforcement Learning • Connectionist • Evolutionary

13. Motivation # 1: Analysis Tool • Suppose that a company have a data base of sales data, lots of sales data • How can that company’s CEO use this data to figure out an effective sales strategy • Safeway, Giant, etc cards: what is that for?

14. Decision Tree induction “if buyer is male & and age between 24-35 & married then he buys sport magazines” Motivation # 1: Analysis Tool (cont’d) Sales data

15. The Knowledge Base in Expert Systems A knowledge base consists of a collection of IF-THEN rules: if buyer is male & age between 24-50 & married then he buys sport magazines if buyer is male & age between 18-30 then he buys PC games magazines Knowledge bases of fielded expert systems contain hundreds and sometimes even thousands such rules. Frequently rules are contradictory and/or overlap

16. Main Drawback of Expert Systems: The Knowledge Acquisition Bottle-Neck The main problem of expert systems is acquiring knowledge from human specialist is a difficult, cumbersome and long activity. KB = Knowledge Base KA = Knowledge Acquisition

17. Motivation # 2: Avoid Knowledge Acquisition Bottle-Neck • GASOIL is an expert system for designing gas/oil separation systems stationed of-shore • The design depends on multiple factors including: • proportions of gas, oil and water, flow rate, pressure, density, viscosity, temperature and others • To build that system by hand would had taken 10 person years • It took only 3 person-months by using inductive learning! • GASOIL saved BP millions of dollars

18. Motivation # 2 : Avoid Knowledge Acquisition Bottle-Neck KB = Knowledge Base KA = Knowledge Acquisition IDT = Induced Decision Trees

19. Full none some no yes waitEstimate? 0-10 >60 30-60 10-30 no Alternate? Hungry? yes Yes no yes No yes Alternate? Reservation? Fri/Sat? yes yes no no yes no yes Raining? No Yes Bar? Yes yes no no yes yes No no Yes Example of a Decision Tree Patrons?

20. Definition of A Decision Tree A decision tree is a tree where: • The leaves are labeled with classifications (if the classification is “yes” or “no”. The tree is called a boolean tree) • The non-leaves nodes are labeled with attributes • The arcs out of a node labeled with an attribute A are labeled with the possible values of the attribute A

21. Databases: what are the data that matches this pattern? Induction: what is the pattern that matches these data? database induction Induction Data pattern

22. Induction of Decision Trees • Objective: find a concise decision tree that agrees with the examples • The guiding principle we are going to use is the Ockham’s razor principle: the most likely hypothesis is the simplest one that is consistent with the examples • Problem: finding the smallest decision tree is NP-complete • However, with simple heuristics we can find a small decision tree (approximations)

23. Induction of Decision Trees: Algorithm • Algorithm: • Initially all examples are in the same group • Select the attribute that makes the most difference (i.e., for each of the values of the attribute most of the examples are either positive or negative) • Group the examples according to each value for the selected attribute • Repeat 1 within each group (recursive call)

24. Example

25. Patrons? full none some X4(+),x12(+), x2(-),x5(-),x9(-),x10(-) X7(-),x11(-) X1(+),x3(+),x6(+),x8(+) Type? burger italian french thai X3(+),x12(+), x7(-),x9(-) X6(+), x10(-) X1(+), x5(-) X4(+),x12(+) x2(-),x11(-) IDT: Example Lets compare two candidate attributes: Patrons and Type. Which is a better attribute?

26. Full none some no yes waitEstimate? 0-10 >60 30-60 10-30 no Alternate? Hungry? yes Yes no yes No yes Alternate? Reservation? Fri/Sat? yes yes no no yes no yes Raining? No Yes Bar? Yes yes no no yes yes No no Yes Example of a Decision Tree Patrons?

27. Decision Trees in Gaming • Black & White, developed by Lionhead Studios, and released in 2001 used ID3 • Used to predict a player’s reaction to a certain creature’s action • In this model, a greater feedback value means the creature should attack

28. Decision Trees in Black & White

29. Decision Trees in Black & White Allegiance Friendly Enemy Defense -1.0 Weak Strong Medium 0.4 -0.3 0.1 Note that this decision tree does not even use the tribe attribute

30. Decision Trees in Black & White • Now suppose we don’t want the entire decision tree, but we just want the 2 highest feedback values • We can create a Boolean expressions, such as ((Allegiance = Enemy) ^ (Defense = Weak)) v ((Allegiance = Enemy) ^ (Defense = Medium))

31. Classification(According to the language representation) • Symbolic • Version Space • Decision Trees • Explanation-Based Learning • … • Sub-symbolic • Reinforcement Learning • Connectionist • Evolutionary