Overview of SRE Project to develop Certification Process for Reliability Assessment Tools

1. Overview of SRE Project to develop Certification Process for Reliability Assessment Tools A Quick Look at a SRE Project that Addresses an Existing Need and Provides a Tangible Product Tyrone Jackson; SRE LA Chapter

2. Summary • Dr. Patrick Larter, President of SRE, requested a plan for reactivating Los Angeles Chapter of SRE • Tyrone Jackson responded to Pat’s request with a proposal for an LA-based SRE working group that would be chartered to standardize and certify commercial and Open Source reliability assessment tools • Mike Canga of NASA Johnson Space Center expressed a strong interest in levying a standard electronic data interchange format (EDIF) for reliability assessment tools that will be used in development of production version of Crew Return Vehicle (CRV) • Standard should provide a single framework for linking different reliability assessment tools

3. Related Back Work • Paper published in 2001 titled: Finding Answers to Space Industry’s Top-10 Reliability Problems • Data collected in 2000 and 2001 identifying Reliability Program problems and causes • Surveyed several reliability engineering experts working for NASA and private sector • Participated in development of a IEEE Reliability Program standard, a IEEE reliability prediction guide, and a SAE FMEA guide (draft) • Evaluated AIAA Collection of Preferred Space-Related Standards (CPSRS) project • Concluded project’s output had little value to reliability community • AIAA cancelled project in 2001 • Evaluated a reliability assessment tool used by NASA in development of prototype version of CRV

4. Cluster of Space Vehicle Failures in 1998-1999 was a Major Problem

5. SRE Project Objective • Objective is to facilitate continuous improvement in capability and utilization of reliability assessment tools used in space industry • Commercial standards have failed to achieve this objective • Several commercial Reliability Program standards are available from IEEE, SAE, and IEC, but decline in effectiveness of Space Systems Reliability Programs has continued since military standards were discontinued in 1994 • Commercial standards provide no guidance for selecting appropriate reliability assessment methods (or tools) for specific objectives of Space Systems Engineering Process • Commercial standards do not define a consistent criteria for implementing or evaluating reliability assessment methods (or tools)

6. SRE Project Plan • Plan is to form a SRE LA Chapter working group that would be chartered to certify compliance of commercial and Open Source reliability assessment tools in accordance with SRE standards • First action for WG would be to develop an Internet-based survey for identifying appropriate reliability assessment methods (and tools) for specific objectives of Space Systems Reliability Programs • Second action for WG would be to cross-reference specific objectives of Space Systems Engineering Process with outputs obtained from reliability assessment methods (and tools) • Third action for WG would be to develop a standard that defines electronic data interchange format (EDIF) for reliability assessment data used in Space Systems Engineering Process • Fourth action for WG would be to begin certifying reliability assessment tools that fully comply with SRE standards

7. Unique Aspect of SRE Project Plan • SRE working group members would develop two sets of certification criteria • One set of criteria for certifying individuals that certify reliability assessment tools • Another set of criteria for use in certifying level of compliance of reliability assessment tool with SRE standards • SRE working group members would be allowed to charge fees for following activities: • Conducting tutorials that explain how to develop reliability assessment tools that comply with SRE standards • Certifying reliability assessment tools

8. Example Survey Question • The affect on Operational Reliability is [ ] if … • The likelihood of this problem is [ ] due to the failure likelihood of one or more of the following key process mitigators… • Rate all applicable risk mitigators: [ ] Funding [ ] Support [ ] Planning [ ] Requirements [ ] Testing [ ] Procedures [ ] Data [ ] Manpower [ ] Coordination [ ] Training [ ] Standards [ ] Guides [ ] Tools [ ] Rules [ ] Knowledge [ ] Experience

9. SAMPLE PLOT OF SURVEY RESULTS

10. Tentative List of Top Five Problems • Reliability critical items often are not identified and controlled early enough  • System reliability predictions often do not include probability of occurrence of all relevant failure modes, failure mechanisms, and failure causes (e.g., probability of damage or induced faults during manufacture, assembly, and test might not be included)  • High-precision reliability prediction values often are not compatible with precision of input data used • Steadily shrinking pool of experienced Reliability Engineering specialist is below level needed to meet critical needs of space industry • Some problems encountered when using commercial reliability assessment tools cannot be solved by a single user (e.g., inability to exchange data between tools, out-of-date input data, non-verifiable data parameters, etc.)

11. Necessary Resources • 8 to 10 reliability engineering experts • 10 or more years experience in space system Reliability Programs • Commits to a 2-hour teleconference/Net Meeting twice per month • Regularly completes working group actions in a timely manner • Obtains employer’s backing • 2 to 4 developers of commercial reliability assessment tools • Provides WG members with fully functional versions of tools to evaluate • Commits to developing tools that comply with standards • SRE national working group chartered to develop a SRE process for approving standards • Commits to developing a process for approving standards by 2003

12. Tentative Schedule • 4 teleconferences (February through March 2002) to develop an Internet survey that identifies appropriate reliability assessment methods based on unique objectives of Reliability Programs in space industry • 8 teleconferences (March through June 2002) to generate a matrix that cross-references objectives of reliability assessment methods with features found in commercial and open source reliability assessment tools • 10 teleconferences (April through August 2002) to develop a standard that defines electronic data interchange format (EDIF) for reliability assessment data used in space industry • Face-to-face meeting (September 2002) in Los Angeles • Standard submitted to SRE for approval (October 2002)

13. Current Progress • Completed tasks: • Drafted electronic survey and posted it in a business web page • Arranged with NASA to used Crew Return Vehicle engineering data to build a guide that cross-references objectives of reliability assessment methods with features found in commercial and open source reliability assessment tools • Pending tasks: • Draft invitation letter for prospective working group members • Generate list of potential volunteers for SRE working group

14. Risk Reduction Actions • What if developers of commercial reliability assessment tools refuse to comply with SRE standards? • SRE should ask NASA for funds and programming support to develop a suit of Open Source reliability assessment tools that fully comply with SRE standards • SRE should maintain Open Source reliability assessment tools on Internet and only charge a fee for shipping self-installable programs on CD • What if developers of commercial reliability assessment tools claim that complying with SRE standards would be too costly • SRE should develop a training program for certifying individuals to make modifications to Open Source reliability assessment tools in compliance with SRE standards • SRE should maintain an Internet database with names of individuals certified to modify Open Source reliability assessment tools

15. Conclusions • This project would significantly enhance SRE’s influence on reliability community and on developers of commercial reliability assessment tools • Concept behind this project is not new but follows current trend of professional societies to create a source of funds through standards and guides • This project can be implemented at no cost to SRE