1 / 20

Enhancing Classification and Regression with Intelligible Generalized Additive Models

This presentation, delivered by Lindsay Stetson at the ACM Conference on Knowledge Discovery and Data Mining, explores the development of accurate and interpretable models in classification and regression tasks. It emphasizes the importance of intelligibility in applied fields such as biology, physics, and medicine, highlighting the use of Generalized Additive Models (GAMs) as a solution. The session includes background motivation, an overview of experimental design, results from various modeling techniques, and conclusions that underscore the performance of bagged and boosted trees in achieving high accuracy while maintaining interpretability.

presley
Download Presentation

Enhancing Classification and Regression with Intelligible Generalized Additive Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intelligible Models for Classification and Regression Yin Lou, Rich Caruana, Johannes Gerhke2012 ACM Conference on Knowledge Discovery and Data Mining Presented by Lindsay Stetson

  2. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  3. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  4. Background • Linear Model • Regression: y = β0+ β1x1 + … + βnxn • Classification: y = logit(β0 + β1x1 + … + βnxn) • Easy to interpret, intelligible, but less accurate • Complex Model (SVM, Random Forest, Neural Networks) • y = (x1, …, xn) • More accurate, but usually unintelligble

  5. Goals of Work “…construct accurate models that are interpretable.” Intelligibility is important! In applied fields like biology, physics, and medicine we need to understand the individual contributions of the features in the model.

  6. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  7. Generalized Additive Model • Regression: y = f1(x1) + … + fn(xn) • Classification: y = logit(f1(x1) + … + fn(xn)) • Each feature gets shaped by a function fi • Goal: Accurate and intelligble

  8. Example

  9. Fitting Generalized Additive Models • Splines (SP) • Single Tree (TR) • Bagged Trees (bagTR) • Boosted Trees (bstTR) • Boosted Bagged Trees (bbTR)

  10. Learning Methods • Least Squares (P-LS/P-IRLS) • Backfitting (BF) • Gradient Boosting (BST)

  11. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  12. Experimental Design

  13. Datasets

  14. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  15. Results

  16. Results

  17. Bias Variance Analysis

  18. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

  19. Conclusion • Generalized additive models are accurate and intelligible • Trees have a low bias but a high variance • Bagging reduces the variance, making the trees methods high performers • Bagged trees, with a low number of leaves, that are gradient boosted are the most accurate

  20. ?

More Related