1 / 28

Statistical Learning & Inference

A comprehensive course on statistical learning and inference covering topics such as supervised and unsupervised learning, regression, classification, model selection, and support vector machines. Includes real-world examples and applications.

teressat
Download Presentation

Statistical Learning & Inference

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Statistical Learning & Inference Lecturer: Liqing Zhang Dept. Computer Science & Engineering, Shanghai Jiao Tong University

  2. Books and References • Trevor HastieRobert TibshiraniJerome Friedman , The Elements of statistical Learning: Data Mining, Inference, and Prediction, 2008, Springer-Verlag • Vladimir N. Vapnik, The Nature of Statistical Learning Theory, 2nd ed., Springer, 2000 • M. Vidyasagar, Learning and generalization: with applications to neural networks, 2nd ed., Springer, 2003 • G. Casella & R. Berger, Statistical Inference, Thomson, 2002 • T. Cover & J. Thomas, Elements of Information Theory, Wiley Statistical Learning and Inference

  3. Overview of the Course • Introduction • Overview of Supervised Learning • Linear Method for Regression and Classification • Basis Expansions and Regularization • Kernel Methods • Model Selections and Inference • Support Vector Machine • Latent Dirichlet Allocation • Unsupervised Learning Statistical Learning and Inference

  4. Intelligent Systems

  5. Why Statistical Learning? • The quiet statisticians have changed our world; not by discovering new facts or technical developments, but by changing the ways that we reason, experiment and form our opinions ....。 –Ian Hacking Statistical Learning and Inference

  6. Problems in Statistical Learning • Predict whether a patient, hospitalized due to a heart attack, will have a second heart attack. • Predict the price of a stock in 6 months from now, on the basis of company performance measures and economic data. • Identify the numbers in a handwritten ZIP code, from a digitized image. • Identify the risk factors for prostate cancer, based on clinical and demographic variables.

  7. Cloud Computing

  8. SARS Risk Age Gender Albumin Blood pO2 RBC Count White Count Chest X-Ray Blood Pressure In-Hospital Attributes Pre-Hospital Attributes ML: SARSRisk Prediction Statistical Learning and Inference

  9. Steering Direction ML: Auto Vehicle Navigation Statistical Learning and Inference

  10. Protein Folding Statistical Learning and Inference

  11. Big Data and AI

  12. EX. Pattern Classification • Objective: To recognize horse in images • Procedure: Feature => Classifier => Cross+Valivation Statistical Learning and Inference

  13. Classifier Horse Non Horse Statistical Learning and Inference

  14. x y S G ^ y LM Function Estimation Model • The Function Estimation Model of learning • Abstract Model: • Generator (G)generates observations x (typically in Rn), independently drawn from some fixed distribution F(x) • Supervisor (S)labels each input x with an output value y according to some fixed distribution F(y|x)

  15. x y S G ^ y LM Function Estimation Model • Key concepts:F(x,y), an i.i.d. k-sample on F, functions f(x,) and the equivalent representation of each f using its index  • Learning Machine (LM)“learns” from an i.i.d. l-sample of (x,y)-pairs output from G and S, by choosing a function that best approximates S from a parameterised function class f(x,), where  is in  the parameter set Statistical Learning and Inference

  16. The Problem of Risk Minimization • The loss functional (L, Q) • the error of a given function on a given example • Therisk functional (R) • the expected loss of a given function on an example drawn from F(x,y) • the (usual concept of) generalisation error of a given function Statistical Learning and Inference

  17. The Problem of Risk Minimization • Three Main Learning Problems • Pattern Recognition: • Regression Estimation: • Density Estimation: Statistical Learning and Inference

  18. General Formulation • The Goal of Learning • Given an i.i.d. k-sample z1,…, zk drawn from a fixed distribution F(z) • For a function class’ loss functionals Q (z ,), with  in • We wish to minimise the risk, finding a function * Statistical Learning and Inference

  19. General Formulation • The Empirical Risk Minimization (ERM) Inductive Principle • Define the empirical risk (sample/training error): • Define the empirical risk minimiser: • ERM approximates Q (z ,*) with Q (z ,k) the Remp minimiser…that is ERM approximates * with k • Least-squares and Maximum-likelihood are realisations of ERM Statistical Learning and Inference

  20. 4 Issues of Learning Theory • Theory of consistency of learning processes • What are (necessary and sufficient) conditions for consistency (convergence of Remp to R) of a learning process based on the ERM Principle? • Non-asymptotic theory of the rate of convergence of learning processes • How fast is the rate of convergence of a learning process? • Generalization ability of learning processes • How can one control the rate of convergence (the generalization ability) of a learning process? • Constructing learning algorithms (i.e. the SVM) • How can one construct algorithms that can control the generalization ability? Statistical Learning and Inference

  21. TRADITIONAL Formulate hypothesis Design experiment Collect data Analyze results Review hypothesis Repeat/Publish NEW Design large experiments Collect large data Put data in large database Formulate hypothesis Evaluate hypothesis on database Run limited experiments Review hypothesis Repeat/Publish Change in Scientific Methodology Statistical Learning and Inference

  22. Learning & Adaptation • Any method that incorporates information from training samples in the design of a classifier employs learning. • Due to complexity of classification problems, we cannot guess the best classification decision ahead of time, we need to learn it. • Creating classifiers then involves positing some general form of model, or form of the classifier, and using examples to learn the complete classifier. Statistical Learning and Inference

  23. Supervised learning • In supervised learning, a teacher provides a category label for each pattern in a training set. These are then used to train a classifier which can thereafter solve similar classification problems by itself. • Such as Face Recognition, Text Classification, …… Statistical Learning and Inference

  24. Unsupervised learning • In unsupervised learning, or clustering, there is no explicit teacher or training data. The system forms natural clusters of input patterns and classifiers them based on clusters they belong to . • Data Clustering, Data Quantization, Dimensional Reduction, …… Statistical Learning and Inference

  25. Reinforcement learning • In reinforcement learning, a teacher only says to classifier whether it is right when suggesting a category for a pattern. The teacher does not tell what the correct category is. • Agent, Robot, Game,…… Statistical Learning and Inference

  26. SL Examples • User interfaces: modelling subjectivity and affect, intelligent agents, transduction (input from camera, microphone, or fish sensor) • Recovering visual models: face recognition, model-based video, avatars • Dynamical systems: speech recognition, visual tracking, gesture recognition, virtual instruments • Probabilistic modeling: image compression, low bandwidth teleconferencing, texture synthesis • …… Statistical Learning and Inference

  27. Course Web • http://bcmi.sjtu.edu.cn/statLearning/ • Teaching Assistant: 顾章轩<zhangxgu@126com> Statistical Learning and Inference

  28. Assignment • To write a short report on the topic you are working on, including: • Problem definition • Model and method • Key issues to be solved • Outcome Statistical Learning and Inference

More Related