E N D
1. Calculating Effect Sizes Definitional effect size formulas
Standardized mean difference
Correlation
Odds ratio
Relating different effect sizes
Calculating d
Calculating r
3. Correlation Effect Size r = standard Pearson product-moment correlation
r does not have a normal distribution
Analyses typically use Z transformation
Variance of Zr:
4. Odds Ratio Effect Size Consider a 2x2 outcome table:
a, b, c, and d represent the counts in each of the cells
Odds ratio = ad / bc
5. Odds Ratio Effect Size Odds ratio does not have a normal distribution, so analyses typically use the natural logarithm of the odds ratio
Variance of ln(odds ratio)
6. Converting Mean Differences to Correlations
7. Converting Correlations to Mean Differences
8. Converting Odds Ratios to Correlations You can compute the tetrachoric correlation from an odds ratio using the formula
where is the odds ratio
9. Converting Correlations to Odds Ratios You can get an approximate value for an odds ratio from a correlation by reversing the prior formula
10. Corrections for Attenuation due to Artificial Dichotomization The measured relationship between two variables will be smaller if either of the variables is artificially dichotomized
To make all of your effect sizes equal, you might choose to correct the measurements for any effect sizes that involved artificially dichotomized variables
11. Formula to Correct for a Single Dichotomization
where p is the proportion of cases in the lower part of the dichotomy, and ?Z is the height of the normal distribution function (not the Z-value) where the cumulative probability is equal to p
12. Calculating Mean Difference Effect Sizes Typically we will calculate the g from each effect, and then later convert them all to ds
Most direct way is to determine , and sp, then directly calculate g
Commonly you do not have these values and so must use alternate methods
13. Calculating g from Between-Subjects Test Statistics Calculating g from t
Calculating g from 1 df F
Using MSE to estimate Sp2
Deriving Sp2 from an F with more than 1 df
Deriving Sp2 from a the F for an unrelated factor that uses the same error term
14. Deriving Sp2 from an F statistic Articles often fail to report the SD’s
However, you can still calculate g if you have the means and at least one F from the ANOVA, even if the F is from a different factor
Fa = MSa/MSe, so MSe = MSa/Fa
Can use MSe as an estimate of Sp2
15. Reconstituting the Error Term The MSe from an ANOVA does not include variance associated with factors included in the design
To calculate g we want to know the naturally occurring variance within our two groups
We can put the variance associated with factors back into the error term
16. Reconstituting the Error Term (cont.) To reconstitute a factor you take the df for the factor and its interactions and add them to the error df, and the SS for the factor and its interactions and add them to the error SS
You usually don’t reconstitute manipulated factors since they are actually artificially adding variance
17. Calculating g from a Within-Subjects Design Within-subjects t tests use a slightly different error term (se-c)
Reasonable to use this instead of sp to calculate g,
Fine if all of your effects used a within-subjects design, but this g will not be comparable to g’s calculated from between-subjects designs
Can convert se-c to sp if you know the correlation between the two measurements
18. Calculating g from a Within-Subjects ANOVA You cannot simply use the overall MSe from an ANOVA as an estimate for se-c when the design includes a within-subjects factor, since different factors use different error terms
se-c can be derived from the error term that would be used to test the factor representing the effect size
19. Calculating g from a Within-Subjects ANOVA (cont.) Shortcut procedure to determine the error terms in a within-subjects design
Once you determine the within-subjects MS for the ANOVA error term, can compute se-c using the following formula
se-c = sqrt (2 * MSwithin error term)
Note that this differs from the formula for between-subjects ANOVA
20. Estimating g from Other Statistics P-values
Unusual statistics that report p-values
Assumed null effects
21. Calculating Correlation Effect Sizes Reports for studies using correlation and regression are more likely to directly report the correlation of interest
Will be much more difficult to derive the correlation if they do not report a direct test of the bivariate relation of interest
Can derive a correlation from any statistic that measures the bivariate relation of interest
22. Calculating Correlation Effect Sizes Multiple regression parameters control for other IVs, so cannot directly be used to derive the correlation
“Tracing method” can be used to get correlation from a multiple regression if the study reports the correlations among the IVs