Effective Verification and Validation Testing

1. Effective Verification and Validation Testing Steve Holt Boeing Commercial Airplanes August 2013

2. EffectiveVerification & Validation Testing Summary - Commercial Airplane Product Development • Initiatives Were Put In Place to Address: • Incomplete or unnecessary test requirements • Immature test capability • Inefficient test conduct • Success found in: • Application of systems engineering tools and methods • Teaming across design and test functions early in the design cycle • Test engineering participation in requirements and V&V development • Questioning why past requirements are needed • Questioning current methods of verification • Focusing on problematic design features and complex integration areas • Optimizing affordability to include risk mitigation (cost avoidance) as well as cost reduction

3. Verification & Validation Cycle Commercial Airplane Product Development Navigate: • Modeling • Simulation • Hardware Bench Testing • Integrated System Testing Full Scale Integrated Testing • Inspection • Analyses • Modeling Verification Planning Requirements Designers Lab Test Value Stream Flight Test Value Stream Start validating the design here!

4. Verification & Validation Target Early Verification and Risk Retirement Typical Plan Target Requirement Verification Analyses Internal Bench Testing Modeling & Simulation Supplier Bench Testing Integrated Systems Laboratory Airplane Flight Test

5. Verification ToolsEducating Design Engineering on Capability Wind Tunnel Full Scale Test Articles Bench Tests Culture Shift Modeling & Simulation Integrated System Testing

6. Verification & Validation Development Cross-Functional Verification, Test, and Means of Compliance • Verification & Validation Governance • Verification Approach • Verification Culture • Verification Capture Interactive Development Among Design and Test Engineers Requirement Satisfying Data • Data Source Determination Verification Plan Integrated Test Plan Test Optimization Test Statement of Work Facility Needs Test System Readiness Triggers Part of test efficiency initiatives Test article readiness ensures early risk mitigation

7. Verification Exercise – Folding Wing Tip • Example of Earlier Verification and Risk Retirement

8. EffectiveVerification & Validation Testing Conclusions • A culture change is often necessary to qualify “Affordability” as cost optimization, to include investment in early verification and risk mitigation • Using test experts to scrutinize design requirements and means of verification leads to a more robust and mature product • Shifting verification “to the left” as much as practical helps ensure a mature and cost-optimized product • Focusing on test readiness and efficiency throughout the development life cycle helps ensure verification success • Complex systems yield emergent effects. Some of them increase risks, some of them increase opportunities.