Improving experiment internal validity: application to measuring the effect of extrinsic information on the perceived qu

1. Quantitative Methods in the Social Sciences (QMSS) Conference, Prague, 20-23 June 2007 Improving experiment internal validity: application to measuring the effect of extrinsic information on the perceived quality of olive oil DEKHILI & d’HAUTEVILLE SupAgro Montpellier (France)

2. Outline Introduction Internal Validity and Satistical Power Method Results

3. Pre-test/post-test designs without control groups Control: one of the fundamental components of experiments (Fisher, 1925; in Shadish et al. 2002) Introduction Experiments: relation of cause and effect (Shadish et al. 2002) In most experiments examining the effect of information on hedonic and sensory perception of food products: the stability of subjects’evaluations is not controled Internal validity ??

4. Objectif of this paper : Use a quasi-experimental design: pre-test/post-test with a control group in an olive oil preference study To detect, in as pertinent a manner as possible, the effect of product-specific regional information on the perception of sensory properties and on perceived quality

5. Statistical Power : Ability to detect the effect of any treatment Can be determined before or after the study Acceptable level: > 0.6 (Verma and Goodale, 1995) Sizes of test and control samples… Internal Validity and Satistical Power Internal Validity : Relevance of control groups Control of external variables Equivalence/non-equivalence between the two groups ANCOVA Model

6. We were unable to obtain 2 samples of equal sizes. We needed a qualitative and tasting information from a large test sample size Method • Consumers (test and control groups) 2 countries (France and Tunisia): 251 consumers Tunisia: - a test group : 107 consumers - a control group: 21 consumers - 13 sessions (1h30mn) France: • a test group : 100 consumers • a control group: 23 consumers - 10 sessions (1h20mn)

7. Condition for recruiting participants : they consume + purchase olive oil • Participant recruiment methods : different according to the context (directly, ads) • Consumers’assignment to the test or control groups on the basis of their availability

8. 4 « extra virgin » olive oils from the same region of production (Vallée des Baux de Provence in France and Zaghouan in Tunisia) • Products and procedure • Information on the dimensions of the image of the region of origin specific to olive oil: was varied • Variety of olive: single variety oil, oil made with a blend of varieties of olive • Natural conditions: soil, climate • Human factor: modern mill, traditional mill • Evaluations in blind condition • Expectation test on the basis of labels • Evaluations in labelled condition

9. Extra virgin olive oil Region of origin Zaghouan Variety of olive Beldi Zaghouan Particular soils Modern mill Extra virgin olive oil Region of origin Zaghouan Blend of varieties of olive Beldi Zaghouan, Chetoui, Sehli Particular soils Traditionnel mill Extra virgin olive oil Region of origin Zaghouan Variety of olive Beldi Zaghouan Localclimate Traditionnel mill Extra virgin olive oil Region of origin Zaghouan Blend of varieties of olive Beldi Zaghouan, Chetoui, Sehli Localclimate Modern mill

10. Effect of the information: • 1st step: Equivalence of the evaluations of the 2 groups in the pre-test • T-test (overall and attribute appreciation, maximum price to pay) • Khi square test (purchase intention) Results • Statistical Power of the experiment prtocol: SamplePower Software : 0.7 in the French case + 0.67 in the Tunisian case Acceptable level (Verma et Goodale, 1995)

11. differences in the hedonic and monetary evaluations : • 2nd step: ANCOVA Model • Variables showing differences between the 2 groups in pre-test: adjust the non-equivalence Scores of the post-test = f (pre-test scores covariate,« group » variable)

12. 3rd step: variables that didn’t show significant differences between the 2 groups in the pre-test Effect of the information: T-test, Khi square test

13. Olive oils Significant effect of the information Tunisia France CM9 Price Price KD2 Price Overall apreciation, price BR5 Price Price HE7 - - We summarise the effect of the region of origin image on the assessment of the olive oils :

14. Thank you for your attention