170 likes | 549 Views
Feedback: The State of Affairs. Assumption: Feedback improves performance.Literature ? AssumptionThe culprit:LACK OF THEORY. What exactly are we talking about?. Feedback = Actions taken by external agent to provide info about some aspect of one's task performanceKRAcross multiple tasks.
E N D
1. The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory Kluger, A.N, & DeNisi, A. (1996)
Psyc Bull, 119, 254-284
2. Feedback: The State of Affairs Assumption: Feedback improves performance.
Literature ? Assumption
The culprit:
LACK OF THEORY
3. What exactly are we talking about? Feedback =
Actions taken by external agent to provide info about some aspect of ones task performance
KR
Across multiple tasks
4. What are we NOT talking about? Does not include:
Natural feedback processes
Task-generated feedback
Person feedback
Feedback-seeking behavior
** Provided from external agent as part of an intervention**
5. Goals To reveal the inconsistent feedback intervention (FI) findings, and disregard for these findings in research;
To quantify the variability of FI effects and address any artifact-based explanations of this variability;
To lay the preliminary foundation for a FI theory, by integrating various theories and empirical findings;
Provide a preliminary test of the FI theory by analyzing hypothesized putative moderators.
6. Finding Studies SSCI, Psycinfo, NTIS
feedback or KR + performance
Abstract and/or title
Back checked refs of previous reviews
3,000+ articles and tech reports
7. Inclusion Criteria The study had to manipulate only the FI
The study had to include a control group or quasi control group that did not receive an FI
Performance had to be measured
The sample had to be of 10 or more participants Either d or other necessary statistics for calculating d had to be provided
Documents in languages other than English, and non-published papers were not considered.
8. Sample 131 studies remained for analyses
607 effect sizes
12,652 participants
23,663 observations
9. General Stats and Considerations Overall sample size weighted mean .41
Variance 0.97
91 effects from single author
17 effects from time-series design
10. FI Theory Development Integration of numerous theories, related constructs and empirical findings
36 potential moderators
FI cues
Task characteristics
Situational variables
Methodological variables
11. Moderators: ES Coding & Data Grad students rated each effect size on each of the 36 moderators
d outliers set to certain value
4 samples from overall sample
all of the data,
potentially dependent data removed
also removing 20 extreme outliers
once also removing the time-series effects.
12. Moderators: Analyses No Q in sight!
(They cite Rosenthal)
Does the moderator variable correlate with d? Type 1 set at .01
If yes, what are values of d for levels of moderator
Remember: run 4 times!
13. Presentation of Results After all exclusions, 470 ES, dbar = 0.38
Variance much lower (0.45)
Presentation of
moderators that were always significant
moderators which became significant
moderators which became insignificant
nonsignificant moderators
14. Presentation of Results Moderators that were always significant:
Discouraging FIs attenuate FI effects
Velocity FIs and
Correct solution FIs augment FI effects
Physical tasks attenuate FI effects
15. Presentation of Results
16. Likes/Dislikes Moderator analyses a little unusual
Variance accounted for?
Rigorous inclusion criteria faith in results
Clear presentation
Running with and without exclusions
Development of theory and theory driven moderators
Great tables and graphs
Real-world application (strict inclusions)
17. Likes/Dislikes File drawer issue
Cultural differences
18. Questions?