1. Basic R 2. Write a Bayesian Inference Function 3. Three Approaches to Bayesian Inference

1. 1. Basic R2. Write a Bayesian Inference Function3. Three Approaches to Bayesian Inference Psychology 548Bayesian Statistics, Modeling & Reasoning Instructor: John Miyamoto 1/15/2014: Lecture 02-2 Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Methods for Computing a Bayesian Statistical Inference • Grid Approximation • based on a finite grid of points in the parameter space. • Conjugate Priors • special classes of functions • MCMC Sampling from theposterior distribution. • BUGS, JAGS, other programs. • Computationally infeasible except if there are only a few parameters. • Computation is too slow. • If a suitable family of conjugate prior distributions exist, this is the best approach. • Conjugate priors only exist in a few special cases. • Very general – many statistical models can be studied this way. Psych 548, Miyamoto, Win '14

3. Bayes Rule – The Simplest Case • H = a hypothesis, e.g., a particular patient has a specific kind of cancer. • = the negation of a hypothesis, e.g., the patient does not have the specific kind of cancer. • Datum = a piece of information, e.g., a medical test result. Psych 548, Miyamoto, Win '14

4. Odds Form of Bayes Rule – Simplest Case Prior Odds(base rate) Likelihood Ratio(diagnosticity) Posterior Odds H = a hypothesis, e.g.., hypothesis that the patient has cancer = the negation of the hypothesis, e.g.., the hypothesis that the patient does not have cancer D = the data, e.g., a + result for a cancer test Psych 548, Miyamoto, Win '14

5. Bayes Rule for Continuous Probability Distributions •  = (1, 2, ....., n) is a vector of parameters. • Θ = {  } is the parameter space Derivation of Odds Form of Bayes Rule for Model Comparison UW Psych 548, Miyamoto, Spr '12

6. Write a Bayesian Inference Function • Demo 02-1. • Vectorized computations Psych 548, Miyamoto, Win '14

7. Methods for Computing a Bayesian Statistical Inference • Grid Approximation • based on a finite grid of points in the parameter space. • Conjugate Priors • special classes of functions • MCMC Sampling from theposterior distribution. • BUGS, JAGS, other programs. • Computationally infeasible except if there are only a few parameters. • Computation is too slow. • If a suitable family of conjugate prior distributions exist, this is the best approach. • Conjugate priors only exist in a few special cases. • Very general – many statistical models can be studied this way. Psych 548, Miyamoto, Win '14

8. Bayesian Approach to a Binomial Inference Problem •  = probability of "success" • We observe N independent trials. X = number of successes. • What is our probability distribution over  given X successes in N trials? The Bayesian Setup • Parameter space Θ = { :   [0, 1] } • Prior probability of : P() over Θ. • The Datum D = "X successes in N trials" Likelihood of D: P(D | , N) = Conjugate Priors UW Psych 548, Miyamoto, Spr '12

9. Grid Approximation for a Binomial Inference Problem • Demo 02-1: See Section 10 and Table 13. • Grid of parameter values:theta = 0.00, 0.02, 0.04, 0.06, ...... , 0.96, 0.98, 1.00 • Data: N = 100 trials; X = 35 successes • Prior Probabilities: All theta are equally likely. E.g., P(theta = 0.00) = 1/51 P(theta = 0.04) = 1/51 P(theta = 0.06) = 1/51 . . . . . . . . . . . . . . . . . . P(theta = 0.98) = 1/51 P(theta = 1.00) = 1/51 Look at Demo02-1Section 10 & Table 13 Psych 548, Miyamoto, Win '14

10. Conjugate Prior • Suppose Ω is a family of probability distributions over Θ = { :   [0, 1] }. This means that for every F  Ω, F is a probability distribution over Θ = { :   [0, 1] } • Definition: The Ω family is a family of conjugate priors for  provided that: if we assume that P() Ω, then P( | D) Ω. • Fact: The beta family of probability distributions is a family of conjugate priors for . Math Formula for a Beta Distribution UW Psych 548, Miyamoto, Spr '12

11. Beta Distributions • A distribution f is a beta distribution if: • NOT IMPORTANT: The mathematical formula • IMPORTANT: If  ~ beta(a, b), then  | X, N ~ beta(a+X, b+N-X) Posterior Probability Distribution Prior Probability Distribution What Does a Beta Distribution Look Like? UW Psych 548, Miyamoto, Spr '12

12. What Does a Beta Distribution Look Like? • See Demo02-1 • Look at the shape of beta(a, b) for different a and b. • Look at shape of prior and posterior for different:a, b = parameters of the betaX, N = results in the data Return to Slide Showing Bayes Rule UW Psych 548, Miyamoto, Spr '12

13. Bayes Rule – Odds Form for Model Comparison M1 & M2 are two different models that predict the data D. Bayes Rule for M1 given D Bayes Rule for M2 given D Odds Form of Bayes Rulefor a Model Comparison Explanation of Odds form of Bayes Rule UW Psych 548, Miyamoto, Spr '12

14. Bayes Rule (Odds Form) in a Model Comparison Prior Oddsis often set to 1.0 Likelihood Ratiois called the “Bayes Factor” Posterior Oddsfor the Models M1 & M2 are two different models that predict the data D. D = the data from a study • A common approach to model comparison is to assume that M1 and M2 have equal prior probability, so the prior odds = 1.0. Then the posterior odds for the models equals the Bayes factor. END UW Psych 548, Miyamoto, Spr '12