Retrospective Evaluation Techniques for Performance Consulting: Theory-to-Practice

1. Retrospective Evaluation Techniquesfor Performance Consulting: Theory-to-Practice Jeff Allen jallen@coe.unt.edu Kim Nimon knimon@coe.unt.edu Russell Elleven relleven@unt.edu

2. When is the last time you learned a new evaluation solution?

3. Retrospective Pretest • Used and promoted in the field of behavioral science research, since the 1950s. • Campbell and Stanley (1963) offered retrospective pretest as a tool to enhance experiment designs.

4. Solution: Retrospective Pretest • Howard (1980) began prescribing this research tool as a remedy for response-shift bias. • Distinguishes itself from the traditional pretest by its relationship to the intervention.

5. When does it work? • Learners don’t know … what they don’t know • There is insufficient time for pre-test. • There as been no benchmarking performed.

6. Why does it work? • It can provide a more accurate measure of actual learning • Provides more differentiation in learning

7. Encompassing Methodology • Pre-Post-Then • Detect response-shift bias • Measure effect of treatment • Control for error • Pre-Then • Measure effect of treatment • Control for error

8. Retrospective Pretest An Example from a law firm: Handout

10. Questions are guaranteed in life . . . Answers aren't

11. Retrospective Evaluation Techniquesfor Performance Consulting: Theory-to-Practice Jeff Allen jallen@coe.unt.edu Kim Nimon knimon@coe.unt.edu Russell Elleven relleven@unt.edu