The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback I

1. Kluger, A.N, & DeNisi, A. (1996) Psyc Bull, 119, 254-284 The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory

2. Feedback: The State of Affairs • Assumption: Feedback improves performance. • Literature ≠ Assumption • The culprit: LACK OF THEORY

3. What exactly are we talking about? • Feedback = • Actions taken by external agent to provide info about some aspect of one’s task performance • KR • Across multiple tasks

4. What are we NOT talking about? • Does not include: • Natural feedback processes • Task-generated feedback • Person feedback • Feedback-seeking behavior ** Provided from external agent as part of an intervention**

5. Goals • To reveal the inconsistent feedback intervention (FI) findings, and disregard for these findings in research; • To quantify the variability of FI effects and address any artifact-based explanations of this variability; • To lay the preliminary foundation for a FI theory, by integrating various theories and empirical findings; • Provide a preliminary test of the FI theory by analyzing hypothesized putative moderators.

6. Finding Studies • SSCI, Psycinfo, NTIS • “feedback” or “KR” + “performance • Abstract and/or title • Back checked refs of previous reviews • 3,000+ articles and tech reports

7. Inclusion Criteria • The study had to manipulate only the FI • The study had to include a control group or quasi control group that did not receive an FI • Performance had to be measured • The sample had to be of 10 or more participants Either d or other necessary statistics for calculating d had to be provided • Documents in languages other than English, and non-published papers were not considered.

8. Sample • 131 studies remained for analyses • 607 effect sizes • 12,652 participants • 23,663 observations

9. General Stats and Considerations • Overall sample size weighted mean .41 • Variance 0.97 • 91 effects from single author • 17 effects from time-series design

10. FI Theory Development • Integration of numerous theories, related constructs and empirical findings • 36 potential moderators • FI cues • Task characteristics • Situational variables • Methodological variables

11. Moderators: ES Coding & Data • Grad students rated each effect size on each of the 36 moderators • d outliers set to certain value • 4 samples from overall sample • all of the data, • potentially dependent data removed • also removing 20 extreme outliers • once also removing the time-series effects.

12. Moderators: Analyses • No Q in sight! • (They cite Rosenthal) • Does the moderator variable correlate with d? Type 1 set at .01 • If yes, what are values of d for levels of moderator • Remember: run 4 times!

13. Presentation of Results • After all exclusions, 470 ES, dbar = 0.38 • Variance much lower (0.45) • Presentation of • moderators that were always significant • moderators which became significant • moderators which became insignificant • nonsignificant moderators

14. Presentation of Results • Moderators that were always significant: • Discouraging FIs attenuate FI effects • Velocity FIs and • Correct solution FIs augment FI effects • Physical tasks attenuate FI effects

15. Presentation of Results

16. Likes/Dislikes • Moderator analyses a little unusual • Variance accounted for? • Rigorous inclusion criteria– faith in results • Clear presentation • Running with and without exclusions • Development of theory and theory driven moderators • Great tables and graphs • Real-world application (strict inclusions)

17. Likes/Dislikes • File drawer issue • Cultural differences

18. Questions?