Exploring Experiments Lab #4: May 2, 2008

1. Exploring ExperimentsLab #4: May 2, 2008 Today’s Article: Goodrick, G.K., Poston, S.C., Kimball, K.T., Reeves, R.S., & Foreyt, J.P. (1998). Nondieting versus dieting treatment for overweight binge-eating women. Journal of Consulting and Clinical Psychology, 66, 363-368.

2. Purpose of Research • Causation • To identify whether the treatment produces change in weight/BMI and/or binge-eating • What might a demonstration study examining this issue look like? • What might an explanation study examining this issue look like?

3. Variables of Interest • Independent Variables (causes) • Dieting treatment: • non-dieting treatment • dieting treatment • wait-list control • Dependent Variables (outcomes) • obesity (operationalized: BMI, weight change) • binge eating (operationalized: BES) • Covariates • Exercise (operationalized: self-report of exercise) • Attendance (operationalized: classes attended)

4. Participant assignment • Random assignment into treatment groups • Dieting group • Non-dieting group • Wait list control group • What is gained by random assignment?

5. Cook and Campbell’s UTOS • Units: Overweight, binge-eating women • Treatment: Non-dieting and Dieting treatments • Observations: Weight and binge-eating habits (also exercise and attendance) • Setting: Not discussed; likely a formal treatment setting for obesity at the Behavioral Medicine Research Center at Baylor University in Houston Texas

6. Threats to Internal Validity • Attrition: 16% of those assigned to conditions dropped out • Authors attempted to lower attrition rates by obtaining a $200 deposit • Authors did conduct Intention to Treat analyses (with the assumption that the missing data values were equal to baseline values) • Compared baseline scores and demographics

7. Threats to Internal Validity • Significant childhood traumas that were influencing eating/exercise/body weight behaviors (at least for some)—could have been included as a covariate. • Hypothetical • Resentful Demoralization: Participants could have been irritated or felt initial defeat if they were assigned to a treatment they felt was less effective. (violates SUTVA). • Compensatory Rivalry: Perhaps the WLC group did so well because they sought an outside treatment. (violates SUTVA).

8. Threats to Construct Validity • Diffusion of treatment • p. 365-366: the diet condition may have been contaminated by non-behavioral modeling factors; “therapist drift” • Inadequate explication of constructs • p. 367: the study used those who scored 21 or higher on the BES, but this cutoff may be too low (i.e., some Ss were not binge eaters); did not measure frequency of binge eating

9. Threats to Construct Validity • Mono-operation bias • Presence of binge eating was assessed only by the BES when other methods are possible • Reactivity effects • E.g., participants may want to present themselves in a favorable light by reporting greater amounts of exercise

10. Threats to External Validity • Results may not generalize to those with a diagnosis of binge eating disorder • Results may not generalize to men • Results may not generalize to those participants who did not meet the screening criteria (e.g., age, BMI, smokers) • Results may not generalize to areas outside of Houston • Results may not generalize to folks who are illiterate or who do not pass by the locations of recruitment advertisements

11. Threats to Statistical Conclusion Validity • Low statistical power: When no significant difference is found between conditions (e.g., no difference between DT and WLC) power is always a concern • Researchers did address power by placing fewer participants in the control group than in the treatment groups • Violation of statistical assumptions: Participants were treated in groups, which may violate the assumption of independent observations

12. Improving the research • Proximal Similarity: Women with higher BES scores would have more closely reflected the “binge-eating” population • Find ways to ensure the integrity of the treatments