Workshop Evaluation Worth the Effort!

1. Workshop EvaluationWorth the Effort! NLTAPA Conference Boise, ID

2. Evaluation Overview • Purpose: To determine the effectiveness of a training program.

3. Evaluation Overview • Four-level model developed by Donald Kirkpatrick (1998). • Reaction • Learning • Behavior • Results

4. Evaluation Overview (continued) • Each successive evaluation level is built on information provided by the lower level.

5. Level 1 • Level 2 • Level 3 • Level 4 • Reaction – Learning – Behavior – Results –

6. Reaction (Level 1) What they thought and felt about the training.

7. Level 1

8. Level 1 • Courses expanded, or condensed • Instructors adjusted • Courses removed from the program pending further review

9. Courses expanded, or condensed

10. Instructors Adjusted

11. Courses Removed

12. Learning – Level 2 Pre and Post Quiz • Developed to evaluate the instruction and content of our training programs (NOT the student) • Consists of 6 questions • Note your answers on the back of the evaluation sheet

13. Quiz Example Question • Denying physical access to someone because of their impairment is: • Sometimes necessary • Discrimination • Permitted by law in certain circumstances • Unimportant compared to making roads safe for cars

14. Learning • Course content modified • Quiz questions refined

15. “Un-Learning” in Winter Maintenance

16. Behavior - Level 3 • Enhanced skills and knowledge identified on the job • Newer process • 6 month evaluation

17. Level 3

18. Cost savings of approximately $5,000 related to remediation of current signage Results – Level 4 • $1,500 of savings in equipment expenditures • $30,000 of expenses avoided that might have been incurred if gas line had to be relocated after completion • Lawsuits avoided

19. Questions ... LTAP – Local Technical Assistance Program Pennsylvania Department of Transportation Bureau of Planning and Research 400 North Street, 6th Floor Harrisburg, PA 17120 • E-mail: LTAP@state.pa.us • Web Site: www.ltap.state.pa.us