Food and Drug Administration Public Meeting: Quality Assurance of Therapeutic Medical Devices in Radiation Oncology

1. Food and Drug Administration Public Meeting: Quality Assurance of Therapeutic Medical Devices in Radiation Oncology Eric Ford, PhD Johns Hopkins University Department of Radiation Oncology and Molecular Radiation Sciences Gaithersburg, MD, June 9-10, 2010

2. Overview • Traditional approaches to QA vs. systems engineering • Recommendations • software changes • error proofing • design of incident reporting systems

3. Quality Assurance in Radiotherapy Traditional approach vs. Systems approach Machine QA Define hazards In-vivo dosimetry Analyze and prioritize Port films Mitigate Chart rounds etc …. Considers specific points Considers the whole chain In the chain and prioritize

4. External Beam Process Map • circa Feb 2009 • 237 nodes • 159 failure modes Consult Simulation Treatment Planning Treatment Ford et al. Int J Radiat Oncol Biol Phys, 74(3), 852-858, 2009

5. Example 1: Wrong scan loaded into TPS RPN score: 168 (RPN=SxOxD = 8x3x7) • How to prevent this error? • QA check: human check before proceeding

6. Checklists • Settling on a standard set of important tasks • Making sure they get done

7. Observations and Recommendations for FDA and Industry • Systems approach to identify weak points • identify where should QA be done (clinic) • identify where improvements are needed (industry) • Integrate QA checks into software • enforced checklists to ensure key steps • e.g., diodes are complete and agree with plan

8. Principles of Error Proofing • Mitigate – control damages • Detect – make problems more obvious • Facilitate – make work easier • Replace – more reliable process • Eliminate – make mistake impossible BETTER

9. Examples of facilitation • Retrain staff • Rewrite policy • Color coding / templates • Standardize processes BETTER

10. Example 1: Wrong scan loaded into TPS RPN score: 168 (RPN=SxOxD = 8x3x7) • How to prevent this error? • QA check • Software double check on demographics

11. Best strategies for error proofing Feasibility Effectiveness

12. Example 2: Treatment plan and DRR (film) pulled up in R&V system Failure mode: Plan pulled up for wrong patient RPN score: 96 Solutions: • Standardize beam names and include course e.g., “CD” • Display patient picture in Tx room • Scan patient ID card just prior to treatment

13. Example 2: Treatment plan and DRR (film) pulled up in R&V system • Patient ID card scanner • Pulls up electronic record • in R&V system • An unanticipated problem • bar code can rub off • Solutions to eliminate this problem? • RFID bracelet • Require changes in software

14. Observations and Recommendations for FDA and Industry • Systems approach to identify weak points • Integrate QA checks into software • Promote (mandate?) industrial-clinical collaboration • rigorous and creative error proofing needed • fully integrate human factors engineers • input from clinic is key • needed early on – end is too late • Regulate carefully. Allow improvement/innovation.

15. The value of incident reporting systems • In-house web-based system available since 2007 • Includes near-misses • 600 “minor incident” for every 1 “critical error” • … Bird and Germain 1996

16. The value of incident reporting systems Incident report logs 8/25/2009 – 11/12/2009 24 total deviations reported FMEA analysis: Jan 2009 – Jun 2009 131 failure modes identified 10/24 (42%) incidents not identified in FMEA • BOTH FMEA and incident reporting needed

17. Incident reporting systems • Large variability month-to-month • Not all staff are using the system

18. Observations and Recommendations for FDA and Industry • Systems approach to identify weak points • Integrate QA checks into software • Promote (mandate?) industrial-clinical collaboration • Regulate carefully. Allow improvement/innovation. • Design incident reporting system carefully • include near misses • use in combination with prospective methods • design goal: easy and usable

19. Observations and Recommendations for FDA and Industry • Systems approach to identify weak points • Integrate QA checks into software • Promote (mandate?) industrial-clinical collaboration • Regulate carefully. Allow improvement/innovation. • Design incident reporting system carefully • Utilize people with experience

20. Acknowledgements Stephanie Terezakis, MD Lee Myers, PhD Ruth Bell, RTT Ted DeWeese, MD Annette Souranis, RTT Danny Song, MD Richard Zellars, MD John Wong, PhD Quality Safety Research Group Johns Hopkins Peter Pronovost, MD, PhD Jill Marsteller, PhD Ayse Gurses, PhD Lilly Engineer, MD Bruce Vanderver, MD