1 / 16

Analysis of Current Maturity Models and Standards

Explore software quality models, maturity standards, and industry practices for risk-based categorization in regulatory compliance for medical devices.

hampson
Download Presentation

Analysis of Current Maturity Models and Standards

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of Current Maturity Models and Standards Robert Sauer Policy Analyst CDRH/OIR

  2. Outline • Software Quality and Maturity • Software Development Standards and Models in Safety Critical Industries • Other Approaches to Risk-Based Categorization • Discussion topics

  3. Software Quality and Maturity • Software quality is an abstract concept • Existing quantitative metrics are incomplete • Initial software quality models defined quality as exhibiting desirable properties (e.g. maintainability, reliability, compatibility, testability, efficiency, ...) • Now, maturity models and standards also consider a firm’s quality management system, processes and procedures to better understand its capability to produce quality software

  4. Quality System Information and Device Regulation • CDRH already uses confidence in the firm’s QMS along with other regulatory controls like adverse event reporting and recalls for risk based classification of medical devices • CDRH already uses knowledge of a certain subset of QS regulations (design controls) to tailor submission requirements for Special 510(k)’s

  5. Initial Working Questions FDA considered • What best practices in software development can be applied to medical devices? • To what extent are these reflected in current standards or maturity models? • What can we learn from standards and models in other safety critical industries? • How can we use that information to realize efficiencies in our review processes for SaMD and SiMD?

  6. Software Development Standards and Models in Safety Critical Industries • Medical devices • IEC 62304 • Other Safety Critical industries • CMMI • DO-178C • IEC 26262 • In development • IEC 82304: Health Software – General Requirements for Product Safety • IEC 33000 Series: Software Process Improvement Capability Determination - SPICE (Formerly 15504)

  7. IEC/ISO 62304 Medical device software –Software life cycle processes • Description • Describes software lifecycle processes and artifacts that should be generated based on device risk class • Results in Conformance/Non-conformance Decision, different requirements based on safety class • Analysis • Aligns very well with CDRH guidance documents • Explicitly includes requirements for safety and security • Designed for and used extensively by medical device manufacturers • Has multiple reputable conformance assessment bodies (e.g. EU) • FDA Recognized • Discussion Topics • How feasibly can 62304 assessments be performed at the firm level? • How do we go beyond 62304? Feasibility of giving credit to those who do more

  8. Capability Maturity Model Integration for Development V1.3 (CMMI) • Description • A collection of best practices that help organizations improve their processes. Originated in software engineering, but has been generalized for any industry. • Levels 2-5: Managed, Defined, Quantitatively Managed, Optimized • Analysis • Aligns well with CDRH guidance documents • Suggests or recommends considerations like safety and security, but does not explicitly include these aspects in the requirements • While CMMI is not designed for a specific industry and currently used by few medical device firms, it is widely used and sometimes required in other safety critical industries like defense, aviation, and aerospace • The Software Engineering Institute, which is the organization that maintains CMMI, has a rigorous certification program for becoming a CMMI appraiser, of which there are many • Discussion Topics • Return on Investment may vary depending on context • Safety is not an explicit requirement; risk is in terms of project risk, not product risk

  9. Aviation • DO-178C Software Considerations in Airborne Systems and Equipment Certification • More detailed about the purpose of each part of the development life cycle and information contained in their associated artifacts • Architectural considerations • Defining and documenting development standards • Specific goals for reviews at each stage of the development process • Normal range and robustness test cases • Tool qualification document

  10. Automotive • IEC 26262 Road Vehicles- Functional Safety • ASIL Decomposition • Mandates or recommends methods and practices, for different ASIL levels • Methods for the verification of software unit design and implementation (Table 9) • Mechanisms for error detection (Table 4) • Structural coverage metrics at the software unit level (Table 12) • Test environments for conducting the software safety requirements verification (Table 16) • Topics to be covered in modelling and coding guidelines

  11. Other Quality and Software Standards • ISO/IEC 12207: Software life cycle processes • ISO/IEC 25000: Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) • IEEE/ISO/IEC 90003: Software Engineering -- Guidelines for the Application of ISO 9001:2008 to Computer Software • ISO 14000 - Environmental management • SAE AS 9100 Quality Systems - Aerospace - Model for Quality Assurance in Design, Development, Production, Installation and Servicing • Nuclear Quality Assurance (NQA-1) Certification • ISO/IEC 17025 (2005)30: General Requirements for the Competence of Testing and Calibration Laboratories • TL 9000 Requirements Handbook (Telecommunications)

  12. Other Approaches to Risk-Based Categorization • Medical device software varies in risk, functionality and complexity • Current CDRH guidance uses Level of Concern (Major, Moderate, Minor) • Many standards tailor requirements based on risk

  13. Risk paradigms: DO-178C

  14. Risk Paradigms: IEC 61508

  15. Risk Paradigms: IMDRF SaMD N12 The four categories (I, II, III, IV) are based on the levels of impact on the patient or public health where accurate information provided by the SaMD to treat or diagnose, drive or inform clinical management is vital to avoid death, long-term disability or other serious deterioration of health, mitigating public health.

  16. Possible Discussion Topics • What would an ideal software maturity or development process evaluation look like? How could it provide additional confidence in product quality? (Goal = provide you the most confidence in the quality of the finished device) • Are there other existing standards that could serve as a starting point? • What are the pros and cons of the standards and models identified? • What lessons can we learn from other industries? • Can a current model be adapted to the medical device industry as is, with an add on, or should we start from scratch? • Are there specific review pain points that these types of evaluation could address? • If as an option we incorporated a model like this in our review, how could we use real-world data to determine if it is working?

More Related