slide1
Download
Skip this Video
Download Presentation
MT2004

Loading in 2 Seconds...

play fullscreen
1 / 70

MT2004 - PowerPoint PPT Presentation


  • 150 Views
  • Uploaded on

MT2004. Olivier GIMENEZ Telephone: 01334 461827 E-mail: [email protected] Website: http://www.creem.st-and.ac.uk/olivier/OGimenez.html. 13. Analysis of variance.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'MT2004' - kylia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

MT2004

Olivier GIMENEZ

Telephone: 01334 461827

E-mail: [email protected]

Website: http://www.creem.st-and.ac.uk/olivier/OGimenez.html

slide2

13. Analysis of variance

  • So far, we’ve investigated the relationship between a response variable and one or several continuous explanatory variables
  • The objective here is to study the relationship between a response variable Y and one or two discrete explanatory variables
slide3

13. Analysis of variance

13.1 One-way ANOVA

  • Example: a standard measurement of the flammability of fabric is given by the length of the burnt portion of a piece of the fabric which has been held over a flame for a given time. An investigation to see whether or not there was a difference between the measurement obtained by 5 laboratories produced the following data.
slide4

13. Analysis of variance

13.1 One-way ANOVA

laboratory

1 2 3 4 5

2.9 2.7 3.3 3.3 4.1

3.1 3.4 3.3 3.2 4.1

3.1 3.6 3.5 3.4 3.7

3.7 3.2 3.5 2.7 4.2

3.1 4.0 2.8 2.7 3.1

4.2 4.1 2.8 3.3 3.5

3.7 3.8 3.2 2.9 2.8

3.9 3.8 2.8 3.2 3.5

3.1 4.3 3.8 2.9 3.7

3.0 3.4 3.5 2.6 3.5

2.9 3.3 3.8 2.8 3.9

Measurements of length obtained by 5 laboratories

slide5

13.1 One-way ANOVA

  • The problem here is to compare several populations
  • The technique we will use is the one-way analysis of variance
  • This is a special case of the ANOVA introduced in the Regression Section
  • Consider k distributions (or populations) with means 1,…,k, and suppose we wish to test:
  • H0: 1=…=k
  • against
  • H0: 1,…,k are not all equal
  • WARNING: the alternative hypothesis does not imply that all the i are different, but at least one pair. E.g. with k = 3, 1=23 would be OK.
slide6

13.1 One-way ANOVA

  • In the example, we wish to test the null hypothesis that the means of lengths obtained by the k = 5 laboratories are the same.
  • Suppose that we have random sample of sizes n1,…,nk from the k distributions. Note that the random samples do not need to have same sample size.
  • yij denotes the jth observation on the ith distribution, i = 1,…, k and j = 1,…, ni
slide7

13. Analysis of variance

13.1 One-way ANOVA

laboratory

1 2 3 4 5

2.9 2.7 3.3 3.3 4.1

3.1 3.4 3.3 3.2 4.1

3.1 3.6 3.5 3.4 3.7

3.7 3.2 3.5 2.7 4.2

3.1 4.0 2.8 2.7 3.1

4.2 4.1 2.8 3.3 3.5

3.7 3.8 3.2 2.9 2.8

3.9 3.8 2.8 3.2 3.5

3.1 4.3 3.8 2.9 3.7

3.0 3.4 3.5 2.6 3.5

2.9 3.3 3.8 2.8 3.9

y23

Measurements of length obtained by 5 laboratories

y57

slide8

13.1 One-way ANOVA

  • In the example, we wish to test the null hypothesis that the means of lengths obtained by the k = 5 laboratories are the same.
  • Suppose that we have random sample of sizes n1,…,nk from the k distributions. Note that the random samples do not need to have same sample size.
  • yij denotes the jth observation on the ith distribution, i = 1,…, k and j = 1,…, ni
  • We will assume that yij is an observation from a random variable Yij where:
  • Yij N(i,2), i = 1,…, k and j = 1,…, ni, Yij independent
  • We thus have that E(Yij) = i
slide9

13.1 One-way ANOVA

  • Actually, this model is a particular case of a multiple regression
  • Define indicator variables x1,…, xk by:
  • Then the equation E(Yij) = i can be rewritten as:
  • E(Yij) = 1x1 + … + kxk
  • This equation defines a multiple regression without intercept 
  • Now, to test the null hypothesis, we can apply the results of the end of the Regression Section (we place equality restrictions on the full model)
slide10

13.1 One-way ANOVA

  • The full model has k parameters (1,…,k) thus p1 = k.
  • The submodel under H0 is E(Yij) = , thus p0 = 1.
  • We have n = n1 + … + nk observations.
  • So an appropriate statistic to test the null hypothesis is:
  • If H0 is false (i.e. 1,…,k are not all equal), then this statistic will tend to take values too large to be consistent with the quantile of a F distribution with k-1 and n-k degrees of freedom.
slide11

13.1 One-way ANOVA

  • We provide other expressions for rss0 and rss1, much easier to manipulate
  • Let denote the overall sample mean
  • Let denote the sample mean of the ith random sample (pop.)
slide12

13.1 One-way ANOVA

  • We provide other expressions for rss_0 and rss_1, much easier to manipulate
  • Let denote the overall sample mean
  • Let denote the sample mean of the ith random sample (pop.)
slide13

13.1 One-way ANOVA

  • We provide other expressions for rss_0 and rss_1, much easier to manipulate
  • Let denote the overall sample mean
  • Let denote the sample mean of the ith random sample (pop.)
slide14

13.1 One-way ANOVA

  • We provide other expressions for rss_0 and rss_1, much easier to manipulate
  • Let denote the overall sample mean
  • Let denote the sample mean of the ith random sample (pop.)
  • It can be shown that the total variability is the sum of the between and within variability:
slide15

13.1 One-way ANOVA

  • It can also be shown that the maximum likelihood are given:
      • For the full model by:
      • For the submodel by:
  • And that:
slide16

13.1 One-way ANOVA

  • If we define:
  • Then
  • Becomes:
slide17

13.1 One-way ANOVA

  • Most often, the sums of squares, mean squares, F values, p-values are displayed in an ANOVA table
  • With
  • Note that the within mean square MSW is an unbiased estimator of the variance 2, called the residual s.e.
slide18

13. Analysis of variance

13.1.1 One-way ANOVA in R

  • Example: a standard measurement of the flammability of fabric is given by the length of the burnt portion of a piece of the fabric which has been held over a flame for a given time. An investigation to see whether or not there was a difference between the measurement obtained by 5 laboratories produced the following data.
slide19

13. Analysis of variance

13.1.1 One-way ANOVA in R

laboratory

1 2 3 4 5

2.9 2.7 3.3 3.3 4.1

3.1 3.4 3.3 3.2 4.1

3.1 3.6 3.5 3.4 3.7

3.7 3.2 3.5 2.7 4.2

3.1 4.0 2.8 2.7 3.1

4.2 4.1 2.8 3.3 3.5

3.7 3.8 3.2 2.9 2.8

3.9 3.8 2.8 3.2 3.5

3.1 4.3 3.8 2.9 3.7

3.0 3.4 3.5 2.6 3.5

2.9 3.3 3.8 2.8 3.9

Measurements of length obtained by 5 laboratories

slide20

13. Analysis of variance

13.1.1 One-way ANOVA in R

  • We wish to test the null hypothesis:
  • H0: 1 = … = 5
  • Against the alternative hypothesis
  • H1: at least one pair of i’s are not equal
  • Where i is the mean length of burnt fabric in measurements from laboratory i (i = 1,…, 5)
slide21

13.1.1 One-way ANOVA in R

> lengthlab1<-c(2.9,3.1,3.1,3.7,3.1,4.2,3.7,3.9,3.1,3.0,2.9)

> lengthlab2<-c(2.7,3.4,3.6,3.2,4.0,4.1,3.8,3.8,4.3,3.4,3.3)

> lengthlab3<-c(3.3,3.3,3.5,3.5,2.8,2.8,3.2,2.8,3.8,3.5,3.8)

> lengthlab4<-c(3.3,3.2,3.4,2.7,2.7,3.3,2.9,3.2,2.9,2.6,2.8)

> lengthlab5<-c(4.1,4.1,3.7,4.2,3.1,3.5,2.8,3.5,3.7,3.5,3.9)

> lab1 <- rep(1,11)

> lab2 <- rep(2,11)

> lab3 <- rep(3,11)

> lab4 <- rep(4,11)

> lab5 <- rep(5,11)

> fabric<-data.frame(lab=c(lab1,lab2,lab3,lab4,lab5),length=c(lengthlab1,lengthlab2,lengthlab3,lengthlab4,lengthlab5))

> plot(fabric$lab,fabric$length)

slide23

13.1.1 One-way ANOVA in R

H0: 1=2=3=4=5

=  ?

slide24

13.1.1 One-way ANOVA in R

> reglab <- lm(length~as.factor(lab), data=fabric)

> reglab

Call:

lm(formula = length ~ as.factor(lab), data = fabric)

Coefficients:

(Intercept) as.factor(lab)2 as.factor(lab)3 as.factor(lab)4

3.33636 0.26364 -0.03636 -0.33636

as.factor(lab)5

0.30909

The lm command produces in that case the parameters estimates of model

E(Yij) = 1x1 + … + 5x5

slide25

13.1.1 One-way ANOVA in R

  • > anova(reglab)
  • Analysis of Variance Table
  • Response: length
  • Df Sum Sq Mean Sq F value Pr(>F)
  • as.factor(lab) 4 2.9865 0.7466 4.5346 0.003337 **
  • Residuals 50 8.2327 0.1647
  • ---
  • Signif. codes: 0 \'***\' 0.001 \'**\' 0.01 \'*\' 0.05 \'.\' 0.1 \' \' 1
  • The R command anova applied to the regression object reglab produces the ANOVA table
  • The pvalue is small, we reject H0 that 1 = … = 5
slide26

13.1.1 One-way ANOVA in R

Checking the assumptions

  • It is crucial to test the assumptions of the ANOVA model, in particular:
  • The observations in each group come from a normal distribution
  • The variances are equal (= 2)
slide27

13.1.1 One-way ANOVA in R

Checking the assumptions

  • It is crucial to test the assumptions of the ANOVA model, in particular:
  • Normality: use QQplot on residuals
  • Homogeneity of variance: inspect the variances
slide28

13.1.1 One-way ANOVA in R

Checking the assumptions

1. The observations in each group come from a normal distribution.

We check normality of the residuals:

> resfab1<-lab1-mean(lab1)

> resfab2<-lab2-mean(lab2)

> resfab3<-lab3-mean(lab3)

> resfab4<-lab4-mean(lab4)

> resfab5<-lab5-mean(lab5)

> resfab<-c(resfab1,resfab2,resfab3,resfab4,resfab5)

> qqnorm(resfab)

> qqline(resfab)

slide29

13.1.1 One-way ANOVA in R

Normality is OK…

slide30

13.1.1 One-way ANOVA in R

Checking the assumptions

2. The variances are equal (2).

We inspect the variances:

> var(lab1)

[1] 0.2045455

> var(lab2)

[1] 0.212

> var(lab3)

[1] 0.138

> var(lab4)

[1] 0.082

> var(lab5)

[1] 0.1867273

Variances are roughly equal

slide31

13. Analysis of variance

13.1.2 Least Significant Differences

  • When performing an ANOVA, we wish to test the null hypothesis:
  • H0: 1 = … = 5
  • Against the alternative hypothesis
  • H1: at least one pair of i’s are not equal
  • So if the null hypothesis is rejected, the question is which differences between groups are most important
  • In other words, we wish to test H0: i = j,  i  j
slide32

13. Analysis of variance

13.1.2 Least Significant Differences

  • An appropriate test to compare the groups in pairs is the 2-sample t-test
  • Under H0: i = j, we have that
  • But 2 is unknown
  • We will replace 2 by s2 = rss1 / (n-k) = MSW (given in the ANOVA table)
slide33

13.1.2 Least Significant Differences

  • On one hand, we have that s2 is independent of
  • On the other hand:
  • So, an appropriate test statistic is:
  • To be compared with the quantile t0.025;n-k for a 2-sided test
slide34

13. Analysis of variance

13.1.2 Least Significant Differences

  • WARNING: This is not exactly the same formula as for the 2-sample t-test since:
  • s2 is calculated using all the data, and not just group i and j
  • the degree of freedom is n - k rather than ni + nj - 2
slide35

13. Analysis of variance

13.1.2 Least Significant Differences

  • If the samples are of equal size, i.e. n1 = … = nk
  • It’s easier to calculate the smallest difference in sample means leading to rejection of the null hypothesis that the 2 groups have equal means
  • This is called the Least Significant Differences (LSD)
  • If k groups, each with m observations (n = mk), the LSD for significance level  is:
  • Once the LSD is calculated, then look for the pairs of groups with sample means differing by more than the LSD
slide36

13.1.2 Least Significant Differences

Example (Fabric data):

The Least Significant Differences for significance level  is:

> LSD <- qt(0.975,55-5)*sqrt(2*0.1647/11)

> LSD

[1] 0.3475762

slide37

13.1.2 Least Significant Differences

> mean(lab5)

[1] 3.645455

> mean(lab2)

[1] 3.6

> mean(lab1)

[1] 3.336364

> mean(lab3)

[1] 3.3

> mean(lab4)

[1] 3

We calculate the sample mean for each group

slide38

13.1.2 Least Significant Differences

> mean(lab5)

[1] 3.645455

> mean(lab2)

[1] 3.6

> mean(lab1)

[1] 3.336364

> mean(lab3)

[1] 3.3

> mean(lab4)

[1] 3

And then look for the pairs of groups with sample means differing by more than the LSD = 0.3475762

slide39

13.1.2 Least Significant Differences

> mean(lab5)

[1] 3.645455

> mean(lab2)

[1] 3.6

> mean(lab1)

[1] 3.336364

> mean(lab3)

[1] 3.3

> mean(lab4)

[1] 3

Suggests that 4 < 2

slide40

13.1.2 Least Significant Differences

> mean(lab5)

[1] 3.645455

> mean(lab2)

[1] 3.6

> mean(lab1)

[1] 3.336364

> mean(lab3)

[1] 3.3

> mean(lab4)

[1] 3

Suggests that 4 < 5

slide41

13.1.2 Least Significant Differences

> mean(lab5)

[1] 3.645455

> mean(lab2)

[1] 3.6

> mean(lab1)

[1] 3.336364

> mean(lab3)

[1] 3.3

> mean(lab4)

[1] 3

Suggests that 4 < 2 and 4 < 5,

but does not suggest any other differences between the i

slide42

13.2 Two-way ANOVA

  • So far, we\'ve considered only one explanatory discrete variable (lab in the fabric data example)
  • Let\'s assume now that each observation belongs to 2 groups
  • This is a 2-way ANOVA
  • Example: consider a reading comprehension test given to pupils of age 9, 10 and 11 from 4 schools (A, B, C and D), giving the scores:
slide43

13.2 Two-way ANOVA

  • Example: consider a reading comprehension test given to pupils of age 9, 10 and 11 from 4 schools (A, B, C and D), giving the scores:
  • So observation/score yi belongs to school j (j = 1,..., J) and age k (k = 1,..., K)
  • yi is an observation of an independent r.v. Yi N(i,2)
slide44

13.2 Two-way ANOVA

  • There are four models of potential interest.
  • Model 3: the expected comprehension score E(Yi) = i is the sum of a school effect and an age effect:
slide45

13.2 Two-way ANOVA

  • There are four models of potential interest.
  • Model 1: the expected comprehension score E(Yi) = i is the result of a school effect only (k = 0, k, k = 1,..., K):
slide46

13.2 Two-way ANOVA

  • There are four models of potential interest.
  • Model 2: the expected comprehension score E(Yi) = i is the result of an age effect only (j = 0, j, j = 1,..., J):
slide47

13.2 Two-way ANOVA

  • There are four models of potential interest.
  • Model 0: the expected comprehension score E(Yi) = i is not the result of a school effect nor an age effect:
slide48

13.2 Two-way ANOVA

  • The ANOVA table for comparing models 0, 1, 2 to model 3 is:
slide49

13.2 Two-way ANOVA

  • The ANOVA table for comparing model 0, 1, 2 and 3 is:

compare model 2 vs model 3

slide50

13.2 Two-way ANOVA

  • The ANOVA table for comparing model 0, 1, 2 and 3 is:

compare model 1 vs model 3

slide51

13.2 Two-way ANOVA

  • The ANOVA table for comparing model 0, 1, 2 and 3 is:

compare model 0 vs model 3

slide52

13.2 Analysis of variance

13.2.1 Two-way ANOVA in R

  • Example: consider a reading comprehension test given to pupils of age 9, 10 and 11 from 4 schools (A, B, C and D), giving the scores:

> score <-c(71,92,89,44,51,85,50,64,72,67,81,86)

> school=c(rep(1,3),rep(2,3),rep(3,3),rep(4,3))

> age=rep(c(1,2,3),4)

> data <- data.frame(school,age,score)

slide53

13.2.1 Two-way ANOVA in R

> data

school age score

1 1 1 71

2 1 2 92

3 1 3 89

4 2 1 44

5 2 2 51

6 2 3 85

7 3 1 50

8 3 2 64

9 3 3 72

10 4 1 67

11 4 2 81

12 4 3 86

slide54

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

slide55

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

Between variability (i.e. between sum of squares and between mean square)

slide56

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

= s2 an unbiased estimate of 2

slide57

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

MSS = SSS / dfS = 1260 / 3

MSA = SSA / dfA = 1256 / 2

slide58

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

FA = MSA / MSB = 628 / 67.67

to be compared with a F2,6;0.025

slide59

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

both school and age effects are significant at the 5% significance level

slide60

13.2.1 Two-way ANOVA in R

# produce the ANOVA table:

> anova(lm(score~as.factor(school)+as.factor(age)))

Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

as.factor(school) 3 1260.00 420.00 6.2069 0.02861 *

as.factor(age) 2 1256.00 628.00 9.2808 0.01458 *

Residuals 6 406.00 67.67

---

Signif. codes: 0 `***\' 0.001 `**\' 0.01 `*\' 0.05 `.\' 0.1 ` \' 1

The observed value of the test statistic of

H0: j = 0 and k = 0 j, k

is ((1260+1256)/5) / 67.67 = 7.43

to be compared with F0.025; 5,6

The p-value is 0.015

slide61

13.2.2 Least Significance Differences

  • If one/two effects are significant, then we\'d like to know which differences between groups (for each effect) are most important
  • Similar reasoning to that employed in the one-way ANOVA gives the  % LSD for the 2 effects:

for factor with J categories

for factor with K categories

slide62

13.2.2 Least Significance Differences

  • For example, as the school factor is significant in the reading comprehension test example, we wish to figure out which schools contribute most
  • The 5% LSD for a difference in mean score between school is

with J = 4 schools and K = 3 age groups

slide63

13.1 Two-way ANOVA

> LSD <- qt(0.975,12-4-3+1)*sqrt(2*67.67/4)

> LSD

[1] 14.23316

slide64

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

We calculate the sample mean for each school

slide65

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

And then look for the pairs of groups with sample means differing by more than the LSD = 14.23316

slide66

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

Suggests that A > B

slide67

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

Suggests that A > C

slide68

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

Suggests that D > C

slide69

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

Suggests that D > B

slide70

13.1 Two-way ANOVA

> mean(score[school==1])

[1] 84

> mean(score[school==4])

[1] 78

> mean(score[school==3])

[1] 62

> mean(score[school==2])

[1] 60

Suggests that A > B, A > C , D > C and D > B

but does not suggest any other differences between the i