Examining the Impact of Narrative Case Studies in Toolbox Talks for Building Construction

1. Examining the Impact of Narrative Case Studies in Toolbox Talks for Building Construction Terri Heidotting, Ed.D. Education and Information Division NIOSH Best Practices in Occupational Safety and Health, Education, Training, and Communication: Ideas That Sizzle Baltimore, Md - October 2002

2. Injury and Fatality Incidence Rates Sources: BLS Census of Fatal Occuptional Injuries and Employment by Industry, 2001 (http://www.bls.gov/news.release/cfoi.t04.htm); BLS Incidence Rates of Nonfatal Occuptional Injuries and Illnesses by Industry Division and Selected Case Types, 1998-2000, Table 7 (http://www.bls.gov/news.release/osh.t07.htm )

3. What are “Toolbox Talks” • Also called “tailgate” talks used in wide range of industries – construction, mining, agriculture • Brief (10-15 minute) weekly safety training sessions (“talks”) • Safety talks conducted on the work site • Safety talk conducted by the site supervisor, foreman, safety supervisor, or senior employee

4. Background • Previous focus groups on improving toolbox talks - Role of narratives / stories • Need for quality training materials that are: • Inexpensive and easily accessible • Have a wide variety of safety topics • Evaluated for their effectiveness • Safety training materials developed with input from the industry • Instructional design of materials grounded in accepted learning theories

5. Training Intervention Effectiveness Research (TIER) • Formative Evaluation: • Determination of training needs • Conceptualization of goals and objectives to meet the needs • Process Evaluation: • Development of draft training materials • Field testing of data collection instruments • Outcome Evaluation: • Controlled study to determine if intended outcomes are obtained and sustained • Identification if critical elements • Impact Assessment: • Longitudinal study - Did training met the educational needs identified in Stage 1 • Examination of the impact of training on the learner and learner’s environment NIOSH Publication Number: 99-142 Website: http://www.cdc.gov/niosh/99-142.html

6. Pilot Study • Draft toolbox lessons • Treatment – with story (narrative) • Control – without story (company’s own lessons) • Data collection and analyses • Focus groups • Interviews • Observations of training sessions • Participants • Electricians, carpenters, laborers with 4 to 38 years of experience • Feedback from instructors and employees

7. Toolbox Talks – Meeting the Needs of Instructors and Employees • Instructors • Uncomfortable presenting materials in front of others • Want to present quality materials that will keep the employees interest and encourage their participation • “Typical toolbox materials are “pretty lame” • “It just goes in one ear and out the other” • Employees “aren’t paying attention…don’t care to be there…they want to sign their name and leave.” • Want materials that are easy to use • Don’t have pre-training preparation time • Employees • Want quality materials that are interesting • “Sometimes it seems like it never happened” • Ineffective toolbox talks are “rushed for time…you’re going to lose your interest right off the bat” • Want topics that reflect work being done • “Get to the point!”

8. Pilot Versions of NIOSH Toolbox Talks for Construction

9. NIOSH Narrative Toolbox TalksWhat Worked • Real-life stories would “get my attention” • Stories that reflect incidents in the employees’ geographical area more interesting • Discussion questions pertaining to the story facilitated participation and interest • Stories made the information easier to understand and remember • Employees and instructors reported they could “relate to” the person in the story • Safety topics reflected work done at the site Real Life Incident— A carpenter and his co-worker were overcome by carbon monoxide (CO) when they entered a basement area where a gasoline-powered generator was lowered into the basement to serve as the source of power. The first worker used a ladder to climb down into the basement to check the generator and was overcome by carbon monoxide. The second worker, who probably entered the basement to assist the downed worker, was also overcome. The workers were dead when they were found and the basement was…….

10. NIOSH Narrative Toolbox TalksChanges Made • Don’t include unnecessary details • Need variety --“Everybody dies” • Bullet additional information for easier manipulation by instructor • Present story discussion questions immediately afterwards • Vocabulary level of text should be simple • Text structured to be read verbatim • Pictures added

11. Revised NIOSH Toolbox Talks for Construction

12. Next Steps... • Formal study – treatment and control groups • Impact of real life cases • Pre-training versus post-training changes in: • Knowledge gains • Safety attitudes • Site observations • Satisfaction with training • Potential differences in injury rates • Instructor feedback • Ongoing improvements in training lessons • Development of public domain materials accessible to all

13. For more information please contact: Terri Heidotting, Ed.D. National Institute for Occupational Safety and Health Education and Information Division 4676 Columbia Parkway, C-10 Cincinnati, Oh. 45226 513/533-8325 1-800-35-NIOSH 1-800-356-4674 toh8@cdc.gov