Creating New Training Evaluation Measures: Subjective Norms And Behavioral Intentions

1. T206 Creating New Training Evaluation Measures: Subjective Norms And Behavioral Intentions By: Mary L. Lanigan, Ph.D. Human Performance & Training Governors State University University Park, Illinois 60466 (708) 534-4051 m-Lanigan@govst.edu

2. Introduction • Introduce 2 instruments that will measure how supportive the work environment is in helping trainees’ transfer their newly learned skills to the job. • This information can potentially provide evidence to show that unsuccessful behavioral change is not due to ineffective training but rather due to an ineffective work environment.

3. Objectives At the end of this lesson you will be able to: • understand how the subjective norms and behavioral intentions measures fit into the Kirkpatrick model; • define what is subjective norms; • define what is behavioral intentions; • understand the origin of these two measures; • create a subjective norms instrument; and, • create a behavioral intentions instrument.

4. Outline This lesson includes the following topics: • Review the levels of the Kirkpatrick model • Discuss Subjective Norms and Behavioral Intentions • Illustrate how to create the two instruments • Practice session were attendees create the measures • Summarize the lesson

5. Review: Kirkpatrick Model Name the 4 levels of the Kirkpatrick model AND describe the instruments typically used at each level. Level 1: Level 2: Level 3: Level 4:

6. Gap In Kirkpatrick Model While the Kirkpatrick model collects data about the trainees, it does not gather information about the work environment. The Problem is: The literature emphasizes the importance of supervisor support in order to transfer skills. If Level 2 data indicates training success but Level 3 is unsuccessful, then who is to blame?

7. Subjective Norms and Behavioral Intentions Definitions • Subjective Norms are people’s perceptions regarding other’s beliefs about whether they should perform the behaviors. • Behavioral Intentions are people’s beliefs about whether they will carry out the behaviors.

8. Subjective Norms and Behavioral Intentions Originates From: The Theory of Planned Behavior (Ajzen, 1985) 1. Attitude 2.Subjective Norms 3. Perceived Behavioral Control Behavioral Intentions Actual Behavior

9. TPB Variables Used To Create A New Theory New Training Evaluation Model * Tier 1: Reactionnaire Tier 2: Attitude and Subjective Norms Tier 3: Perceived Behavioral Control Factors Tier 4: Behavioral Intentions and Actual Behavior Tier 5: Results and Return-On-Investment * See Reference title: How To Create Evaluation Instruments To Predict Behavior Transfer: A New Theory And Measures In Training Evaluation

10. Using The Instruments With Kirkpatrick Model To fill the gap in the Kirkpatrick model, these measures then serve the following purposes: Subjective Norms: to assess the amount of perceived support coming from personnel who directly impacts trainees’ performances. Behavioral Intentions: to assess trainees’ intentions to carry out the newly learned behaviors on-the-job.

11. Creating The Instruments To illustrate how to create these instruments, let’s use the following situation as an example: Context: A teller training class. It’s a 3 week session. The trainees’ behaviors are influenced by many people. The trainees are expected to immediately transfer their newly learned skills to the job.

12. Creating A Subjective Norms Instrument • Determine the individual(s) that influence trainees’ behaviors on-the-job. • Determine which training performance objectives you wish to measure. • Take the performance part of each objective and turn it into an item for the instrument. • Make sure each item is written in the neutral. • Make sure each item is a single behavior. • Create a scale of responses.

13. Creating A Behavioral Intentions Instrument • Determine which training performance objectives you wish to measure. • Take the performance part of each objective and turn it into an item for the instrument. • Make sure each item is written in the neutral. • Make sure each item is a single behavior. • Create a scale of responses. • Make sure the behavioral intentions measure parallels the actual behavior instrument.

14. AdministeringThe Instruments Administer both measures pre and post training. • To determine if training has created change. • If post scores are significantly higher than your evidence that training is effective is better substantiated. • If there are indications of significant problems on the pre measure, then you can correct the problems prior to the trainees returning to the environment.

15. Practice Exercise Turn to the practice exercise on the next page.

16. References Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl & J. Beckmann, (Eds.), Action- control: Fromcognition to behavior (pp. 11-39). Heidelberg: Springer. Kirkpatrick, D.L. (1994). Evaluating training programs: The fourlevels. San Francisco: Berrett-Koehler. Lanigan, M. (in print). How to create evaluation instruments topredict behavior transfer: A new theory and measures intraining evaluation. Chicago: Third House.