University of Southern California Department Computer Science

1. Bayesian Logistic Regression Model (Final Report) Graduate Student Teawon Han Professor Schweighofer, Nicolas 9/23/2011 University of Southern California Department Computer Science

2. Bayesian Logistic Regression Model (Final Report) The purpose of the project - Experiment ? 2. Summary of Bayesian Logistic Regression (BLR) - How do I apply BLR to the BART or ART 3. What is next? • Introduction PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

3. Bayesian Logistic Regression Model (Final Report) • Predict accurate status of rehabilitation • - Reduce rehabilitation time ( No un-necessary training ) • - Rise efficiency in rehabilitation process • Data Collection method • - use 3 days data in my program (regression) for test • The purpose of the project PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

4. Bayesian Logistic Regression Model (Final Report) • Experiment Environment • The purpose of the project ` Success! PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

5. Bayesian Logistic Regression Model (Final Report) Success condition 4. Given Data type (collected data 150) Error ==0 && Hit Hand ==1 • The purpose of the project Data (1 day) New New Weight value Prior Weight value New Weight value Pattern analysis Pattern analysis Data (2 day) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

6. Bayesian Logistic Regression Model (Final Report) • What is regression? Why do we use regression? • Example ( Linear Regression ) • Regression can help • to represent • complete model by • partially observed • data. • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

7. Bayesian Logistic Regression Model (Final Report) • How do I apply BLR to the project • - First, we have two classes for classification. • ( Success and Fail ) • - Expression • a. p(C1 | Ф ) = y (Ф) = Ϭ (WT Ф)  success • b. p(C2 | Ф ) = 1 - p(C1 | Ф )  fail • where Ф is feature vector ( data ) and w is weight vector. • Summary of Bayesian Logistic Regression (BLR) Success condition Error ==0 && Hit Hand ==1 PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

8. Bayesian Logistic Regression Model (Final Report) • How do I apply BLR to the project (continue) • - Second, to represent Logistic Regression, I used Ϭ(·). • where Ϭ(α) = 1 / 1 + exp (-α) • a. range is limited • (0 ~ 1) • b. TO MAKE EASY, • I used simplest • formula • (next page) • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

9. Bayesian Logistic Regression Model (Final Report) • How do I apply BLR to the project (continue) • b. TO MAKE EASY, I used simplest formula which • includes the least number of parameters (features) • Formula : W0 + W1Ф1 +W2Ф2 •  this should be updated more accurately by • Nuero-Scientific knowledge. • 4. The goal in here is ‘Finding accurate W vector’ to predict posterior result. (next page) • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

10. Bayesian Logistic Regression Model (Final Report) • The goal in here is ‘Finding accurate W vector’ to predict posterior result. • - Process of calculation W vector • (W can be represented by Gaussian) • a. Wmap (mean) SN (covariance) • : Wmap can be calculated by Newton-Raphson rule. • b. Newton-Raphson rule • : Iterative Optimization Scheme to make minimize • the error of weight vector. [link] • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

11. Bayesian Logistic Regression Model (Final Report) • The goal in here is ‘Finding accurate W vector’ to predict posterior result. • - Process of calculation posterior W vector • c. Equation of Newton’s method (Wmap ) • ( Pattern Recognition and machine learning book • p208 ) • d. Covariance of W • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

12. Bayesian Logistic Regression Model (Final Report) • The goal in here is ‘Finding accurate W vector’ to predict posterior result. • - Process of calculation W vector • e. Finally, we can get distribution of posterior W • To get the posterior probability given data with posterior W • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

13. Bayesian Logistic Regression Model (Final Report) • To get the posterior probability given data with posterior W (derivation) • - you can find • “Pattern recognize and machine learning book” • - I also attached from Srihari’s lecture note. • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

14. Bayesian Logistic Regression Model (Final Report) • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

15. Bayesian Logistic Regression Model (Final Report) • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

16. Bayesian Logistic Regression Model (Final Report) • Summary of Bayesian Logistic Regression (BLR) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

17. Bayesian Logistic Regression Model (Final Report) • How do I apply BLR to the project • a. Initial weight vector = [0.001,0.001,0.001] • b. Initial covariance vector = [1,0,0 ; 0,1,0; 0,0,1] • Summary of Bayesian Logistic Regression (BLR) Data (1 day) New New Weight value New New Prior Weight value New Weight value Pattern analysis Pattern analysis new Data (2 day) PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA

18. Bayesian Logistic Regression Model (Final Report) • 7. Results • Summary of Bayesian Logistic Regression (BLR) predict predict PRESENTED BY TEAWON HAN Of UNIVERSITY OF SOUTHERN CALIFORNIA