How to calculate: - Average Effect Size (step 4) - Moderators (step 5)

1. How to calculate: - Average Effect Size (step 4) - Moderators (step 5)

2. Overview of PPT slides • We will first go through “how” to do this. • Then, we will repeat everything and explain “why” we do each step

3. Overview of Data Analysis • Central tendency – Effect size • What is the average effect and is it significant? • Variability – Homogeneity test • Does average affect have variability? (If so, then you test moderators) • Prediction • Does the average effect differ with moderators?

4. Average effect size (ES) • Overview • Conceptually… • First, transform “r” into Fisher’s z-to-r • Second, weight them by sample size/inverse variance • Third, sum them together • Fourth, divide by sum of total sample size. • In Practice… • You transform “r” into Fisher’s z-to-r • You calculate inverse variance • Then use Macro to do the rest

5. At this point you have: (1) Effect sizes. (2) Sample sizes. (3) moderators

6. Step 1 - Transform “r” into Fisher’s z-to-r • See formulas from “Example-DataSet2”

7. Step 2 - Weight by inverse variance • See formulas from “Example-DataSet2”

8. Step 3 - Upload to SPSS • in SPSS, file-open

9. Step 4 - Initiate “MeanES” macro • Download from Lipsey/Wilson website • Open YOUR DATASET • Open a new syntax • Put the following at the end of the syntax file • INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MeanES.SPS' . • Highlight the sentence • Click run (blue triangle at top)

10. Step 5 - Calculate “MeanES” • In syntax, type the following sentence: • MEANES ES = ES_zr /W = weight /PRINT IVZR. fyi – “ES_zr” is my name for the effect sizes “weight” is my name for the weight “/PRINT IVZR” converts output back to “r” • Highlight and click run

11. Step 6 – Interpret output • ES = .1225, p = .0000 • Q = 1140.11, p = .0000

12. Now, let’s repeat and explain “why” • Transform into Fisher’s z-to-r • Why? “r” does not have a normal distribution because it is skewed at the tails. Fishers’ z-to-r has a normal distribution, so it is the preferred metric. • Weight by inverse variance • Why? The larger the sample size of a particular study, the larger the impact. Weighting by the inverse takes into account sample size. • Control for standard error • Why? When you weight by the inverse variance, then the standard errors are incorrect, so you can’t use the traditional tests within SPSS. The macros by Lipsey/Wilson control for the problem with standard error.

13. Now, let’s move to Moderators • Overview • Conceptually… • First, need to ascertain “homogeneity” which tells you if variance exists in average effect size • Can test categorical moderators (categories like college student versus actual juror) similar to ANOVA • Can test continuous moderators (such as length of stimulus) similar to Regression • In Practice… • Use macros • Macros exist for ANOVA & Regression

14. Macros for moderators • METAF – Categorical Moderators • INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MetaF.SPS' . • METAF ES = ES_zr /W = weight /GROUP = ev1_subjecttype /PRINT IVZR. • METAF – Continuous Moderators • INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MetaREG.SPS' . • METAREG ES = ES_zr /W = weight /IVS = ev1_subjecttype /PRINT IVZR. fyi – can only run 2 macros in same session fyi – I run categorical moderators using both METAF and METAREG because answer different questions

15. Interpreting Categorical Macro • Sig difference = 22.87, p = .0000

16. Interpreting Continuous Macro • beta = -.0958, p = .0014

17. Now, to Multivariate • METAREG can handle multiple variables • METAREG ES = ES_zr /W = weight /IVS = ev1_subjecttype ev2_stimulustype /PRINT IVZR. fyi – Can include as many variables as you wish. fyi - I believe you can include CATEGORICAL moderators IF: • they are dichotomous, • they are continuous and linear relationship

18. Interpreting Continuous Macro • beta for each one, p value for each one • overall r-squared

19. Finally, Interaction analysis • Center each variable • Create interaction term by multiplying together • Enter all three into METAREG • If interaction term is sig, then interaction exists • How to graph the interaction? (next week)

20. FYI • Our website has my excel file (Example-DataSet2) and the accompanying SPSS file (SPSS-ExampleDataSet2) • You also have my quals paper, so you can use the SPSS file to practice and see if your data match the quals paper. • HOWEVER, the data will only match the CATEGORICAL moderator analysis (not the continuous moderator analysis) for reasons I don’t have time to go into.