1 / 18

Geraldine Aglipay, BS and Karen Peters, DrPH

Kirkpatrick model of evaluation to assess public health workforce preparedness training American Public Health Association November 5, 2013 Boston, MA. Geraldine Aglipay, BS and Karen Peters, DrPH Illinois Preparedness and Emergency Response Learning Center

elmo
Download Presentation

Geraldine Aglipay, BS and Karen Peters, DrPH

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kirkpatrick model of evaluation to assess public health workforce preparedness trainingAmerican Public Health AssociationNovember 5, 2013Boston, MA Geraldine Aglipay, BS and Karen Peters, DrPH Illinois Preparedness and Emergency Response Learning Center MidAmerica Center for Public Health Practice University of Illinois at Chicago, School of Public Health

  2. Presenter Disclosures Karen E. Peters “No relationships to disclose” (1) The following personal financial relationships with commercial interests relevant to this presentation existed during the past 12 months:

  3. Overview: IPERLC • 1 of 14 Preparedness and Emergency Response Learning Centers (PERLC) • Funded by CDC’s Office of Public Health Preparedness and Response • Provides education, training and technical assistance services to enhance capacity/performance of IL public health workforce regarding preparedness and emergency response

  4. Overview: Mini-Grants for Emergency Preparedness • To increase emergency response capability and capacity in Illinois • To support short-term, targeted preparedness related activities for which there was a need but no discretionary resources available • To gauge the efficacy of individual training on organizations, their staff and partners (e.g., fire depts, LTC facilities)

  5. Overview: Mini-Grants for Emergency Preparedness • Grant awards: Up to $4,500 for 6 month projects (March 1-August 30, 2012) • Eligibility/Awardees (n=13): • Local health departments [n = 10] • Primary care clinics (FQHCs or Stand alone) [n=2] • CBOs (NFPs, Schools, FBOs) [n=1]

  6. Overview: Mini-Grants for Emergency Preparedness • Project descriptions: • Conduct training activities; Run exercises; Develop response plans; Conduct needs assessments; Create resources; Build support networks • Audiences and Partners (at risk): • Disabled persons, rural, seniors, economically disadvantaged, isolated, infants/children, limited/non-english speaking persons

  7. Overview: Evaluation using Kirkpatrick Model • Adopted by CDC for PERLC evaluation, organized as 4 levels: • Level (1) Trainees' reaction • Level (2) Knowledge acquired, skills improved, or attitudes changed • Level (3) Practice behavior changed (field) • Level (4) Results, or impact (at agency/organizational level)

  8. The Issue • Little literature on the intermediate and long term impacts of preparedness and emergency response training on agency capacity to respond to emergencies • Focus is generally on individual KSA capacity (knowledge, skills, attitudes)

  9. The Level 3 & 4 Grantees • IPERLC selected 3 grantee LHDs based on training content and stakeholders (LTCF) for participation in Level 3 & 4: • 1 rural, 1 suburban, 1 urban • All 3 sites completed Level 1 & 2 • Level 3 evaluations (6,12 months) at 2 sites • Level 4 evaluations (at 12 months) at all 3 sites

  10. Results: Level 3 • Self report increased strength and confidence in ability to apply training lessons to field experience • Work policy changes help with work preparedness • Many semi- or fully retired, so future training delivery/content needs to be considered for their needs [attrition]

  11. Results: Level 3 • “additional training on policies and procedures needed”; • “more exercises”; • “trained people more willing to assist with other activities as they realize that preparedness is about team work”

  12. Results Level 4 • Partnerships have been strengthened [LTCF, special needs populations (4.6/5.0) • Training outcomes meet organization’s needs/responsibilities for preparedness (4.0/5.0)

  13. Results Level 4 • New resources brought into the partnership • 2 of 3 report new policies developed • In absence of minigrant training, not able to accomplish same outcomes

  14. Results Level 4 • ‘Continued relationship building with special needs and LTC facilities…not historically involved with preparedness’ • ‘We grow and expand relationships with partners and staff are more aware of their responsibilities’ • ‘A foundation to prepare for full scale evacuation all LTCF participated and better able/quite confident to respond to emergencies’

  15. Discussion • Level 3 and 4 labor intensive • Can do Level 4 (retrospectively) without other levels to see organizational changes but don’t have resources to conduct all 4 Kirkpatrick levels but… • Requires key informant interviews, document policy reviews not only with primary public health agencies, but their top 1-3 key partners

  16. Conclusions • Evaluation of (preparedness) training using Kirkpatrick methodology provides guidance but is labor intensive to implement • Instability of funding for preparedness training and personnel changes at local level make comprehensive evaluation difficult

  17. Contact Information • Victoria Wiebel, Assistant Director • Illinois Preparedness and Emergency Response Learning Center • vikki@uic.edu • http://www.midamericacphp.com/iperlc/

  18. Thank you!

More Related