The National Human Services Training Evaluation Symposium: - A 10 Year Retrospective -

1. The National Human Services Training Evaluation Symposium: - A 10 Year Retrospective -

2. 10 Yrs of Picturesque Settings The Faculty Club

3. 10 Years of Enlightening Presentations

4. Looking back and moving forward……..

5. 1998: The First In the beginning

6. 1999: The 2nd Annual • Keynote: Bernadette McCarthy—Using Training Evaluation to Improve Services • Program Examined: • Training outcomes: performance monitoring • Delphi Techniques • Nominal Group Techniques • Knowledge testing • Promoting & evaluating skill development • Using training to influence policy • Levels of competence • TOL

7. 2000: The 3rd Annual Kirkpatrick’s model, but what about Level Four? • Keynote: Icebreaker on the first evening instead • Program examined: • Evaluation of Interdisciplinary training: Child Welfare workers and supervisors • Strategic training evaluation • Embedded evaluation • TOL/measuring behavioral change • Skills assessment

8. 2000

9. 2001: The 4th Annual The year of “Affinity Action Groups” • Keynote: Dave Wegenast on “Return on Investment” of the Evaluation Symposium • Program examined: • Assessing underlying competencies • Models for evaluating core curriculum • Evaluation of statewide training program • The logic model • Integration of TOL • Evaluating group behavior • Blackboard—E Learning

10. 2001

11. 2002: The 5th Annual Outcomes in Child Welfare • Keynote: Mark Courtney–Evaluation in Child Welfare Services: Big Questions • Program examined: • Training Evaluator Competency • SPrA • CPS system • Compliance with practices standards • Needs assessments

12. 2002

13. 2003: The 6th Annual Measuring outcomes, the long and winding road… • Keynote: Elena Cohen—Getting the whole picture: The CFSR, Training and Frontline Practice Links • Program Examined: • Chain of evidence • Panel presentation on each of Kirkpatrick’s levels • Project briefings

14. 2003

15. 2004: The 7th Annual Improving Evaluation Capabilities • Keynote: Wayne Stevenson—Training as an Avenue to Support Practice Improvement • Program Examined: • National Frameworks: National Evaluation of Child Welfare Training & North American Certification Project • TOL • Staff Retention • Organizational Culture

16. 2004

17. 2005: The 8th Annual Expanding our horizons: Enhancing current methods, creating new methods… • Keynote: Eileen Gambrill • Program Examined: • Future trends in child welfare training • Addressing fairness & equity in training evaluation • Training evaluation in large systems • Linking training outcomes to children and families • Instruments to measure skill acquisition • Practice assessments in a large system

18. 2005

19. 2006 – The International House

20. 2006: The 9th Annual Evidence-based training: What does it mean when we say training is effective? • Keynote: Kurt Kraiger – Alternative Frameworks for Training Evaluation • Program Examined: • How evaluation brings attention to gaps in training • Teaching & training methods that contribute to knowledge retention • Logic models • Measuring knowledge/skill • Evaluating assumptions, values and beliefs

21. 2006

22. 2007: The Berkeley City Club

23. 2007: 10th Annual Looking back and moving forward… • Keynote: Peter Watson—Training Evaluation as a tool for organizational development • Program Examines: • Findings from the National Evaluation of Child Welfare grants • Measuring TOL • Revisiting Level 1 • The future of training evaluation • Using tests to predict job performance • Evaluation of training content

24. Lessons Learned…(NSDTA Journal Article) • You can teach a turkey to climb a tree, but it's easier to hire a squirrel. • Effective utilization of training is influenced by individual, organizational, and training design factors. • Training evaluation can help administrators differentiate between training and non-training solutions.

25. Lessons Learned, cont’d 4. We know what we don’t know. 5. Sometimes the training evaluation “tail” wags the training content “dog”. 6. Be careful what you wish for, the focus on outcomes cuts both ways.

26. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

27. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • Increased funding, value, and visibility of training and development

28. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • Increased funding, value, and visibility of training and development • Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career

29. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • Increased funding, value, and visibility of training and development • Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career • Increased accountability of child welfare/human services

30. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • Increased funding, value, and visibility of training and development • Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career • Increased accountability of child welfare/human services • Introduction of litigation into the HSTD arena

31. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

32. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level • Technology – An increase in distance models and alternative delivery systems

33. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level • Technology – An increase in distance models and alternative delivery systems • Agency-University partnerships

34. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT • An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level • Technology – An increase in distance models and alternative delivery systems • Agency-University Partnerships • Development of the field and emerging profession of HSTD (e.g., Code of Ethics, roles and competencies, journal, landmark events 10 years NHSTES & 20 years NSDTA conference)