The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback I

1. The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory Kluger, A.N, & DeNisi, A. (1996) Psyc Bull, 119, 254-284

2. Feedback: The State of Affairs Assumption: Feedback improves performance. Literature ? Assumption The culprit: LACK OF THEORY

3. What exactly are we talking about? Feedback = Actions taken by external agent to provide info about some aspect of one�s task performance KR Across multiple tasks

4. What are we NOT talking about? Does not include: Natural feedback processes Task-generated feedback Person feedback Feedback-seeking behavior ** Provided from external agent as part of an intervention**

5. Goals To reveal the inconsistent feedback intervention (FI) findings, and disregard for these findings in research; To quantify the variability of FI effects and address any artifact-based explanations of this variability; To lay the preliminary foundation for a FI theory, by integrating various theories and empirical findings; Provide a preliminary test of the FI theory by analyzing hypothesized putative moderators.

6. Finding Studies SSCI, Psycinfo, NTIS �feedback� or �KR� + �performance Abstract and/or title Back checked refs of previous reviews 3,000+ articles and tech reports

7. Inclusion Criteria The study had to manipulate only the FI The study had to include a control group or quasi control group that did not receive an FI Performance had to be measured The sample had to be of 10 or more participants Either d or other necessary statistics for calculating d had to be provided Documents in languages other than English, and non-published papers were not considered.

8. Sample 131 studies remained for analyses 607 effect sizes 12,652 participants 23,663 observations

9. General Stats and Considerations Overall sample size weighted mean .41 Variance 0.97 91 effects from single author 17 effects from time-series design

10. FI Theory Development Integration of numerous theories, related constructs and empirical findings 36 potential moderators FI cues Task characteristics Situational variables Methodological variables

11. Moderators: ES Coding & Data Grad students rated each effect size on each of the 36 moderators d outliers set to certain value 4 samples from overall sample all of the data, potentially dependent data removed also removing 20 extreme outliers once also removing the time-series effects.

12. Moderators: Analyses No Q in sight! (They cite Rosenthal) Does the moderator variable correlate with d? Type 1 set at .01 If yes, what are values of d for levels of moderator Remember: run 4 times!

13. Presentation of Results After all exclusions, 470 ES, dbar = 0.38 Variance much lower (0.45) Presentation of moderators that were always significant moderators which became significant moderators which became insignificant nonsignificant moderators

14. Presentation of Results Moderators that were always significant: Discouraging FIs attenuate FI effects Velocity FIs and Correct solution FIs augment FI effects Physical tasks attenuate FI effects

15. Presentation of Results

16. Likes/Dislikes Moderator analyses a little unusual Variance accounted for? Rigorous inclusion criteria� faith in results Clear presentation Running with and without exclusions Development of theory and theory driven moderators Great tables and graphs Real-world application (strict inclusions)

17. Likes/Dislikes File drawer issue Cultural differences

18. Questions?