1 / 26

Decision Trees

Decision Trees. Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Modified by Longin Jan Latecki Some slides by Piyush Rai. Outline. Decision Tree Representations ID3 and C4.5 learning algorithms (Quinlan 1986) CART learning algorithm (Breiman et al. 1985)

ridley
Download Presentation

Decision Trees

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decision Trees Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Modified by Longin Jan Latecki Some slides by Piyush Rai Decision Trees

  2. Outline • Decision Tree Representations • ID3 and C4.5 learning algorithms (Quinlan 1986) • CART learning algorithm (Breiman et al. 1985) • Entropy, Information Gain • Overfitting Decision Trees

  3. Training Data Example: Goal is to Predict When This Player Will Play Tennis? Decision Trees

  4. Decision Trees

  5. Decision Trees

  6. Decision Trees

  7. Decision Trees

  8. Learning Algorithm for Decision Trees What happens if features are not binary? What about regression? Decision Trees

  9. Choosing the Best Attribute A1 and A2 are “attributes” (i.e. features or inputs). Number + and – examples before and after a split. - Many different frameworks for choosing BEST have been proposed! - We will look at Entropy Gain. Decision Trees

  10. Entropy Decision Trees

  11. Entropy is like a measure of impurity… Decision Trees

  12. Entropy Decision Trees

  13. Decision Trees

  14. Information Gain Decision Trees

  15. Decision Trees

  16. Decision Trees

  17. Decision Trees

  18. Training Example Decision Trees

  19. Selecting the Next Attribute Decision Trees

  20. Decision Trees

  21. Non-Boolean Features • Features with multiple discrete values • Multi-way splits • Test for one value versus the rest • Group values into disjoint sets • Real-valued features • Use thresholds • Regression • Splits based on mean squared error metric Decision Trees

  22. Hypothesis Space Search • You do not get the globally optimal tree! • Search space is exponential. Decision Trees

  23. Overfitting Decision Trees

  24. Overfitting in Decision Trees Decision Trees

  25. Validation Data is Used to Control Overfitting • Prune tree to reduce error on validation set Decision Trees

  26. Homework • Which feature will be at the root node of the decision tree trained for the following data? In other words which attribute makes a person most attractive? Decision Trees

More Related