Design For Variation

1. Design For Variation UCM 2012 Sheffield, UK July 2-4, 2012 Grant Reinman, Senior Fellow, Statistics and Design For Variation Pratt & Whitney, East Hartford, CT

2. Pratt & Whitney EngineeringA Passion for Innovation PurePower® PW1000G Engine

3. Deterministic Design, Uncertain WorldTraditional Approach: Empirical Design Margins, Factors of Safety • Usage • Manufacturing • Computational Models • Materials

4. Probabilistic Design, Uncertain WorldWhy? To Help Prevent Design Iterations due to a Model’s • Meanline Miss, by using Bayesian model calibration process • Margin Miss, by replacing legacy margins with a probabilistic model of uncertainty and variability • To Reduce Cost • Focus on important features • Relax requirements on unimportant features • Use Robust Design to reduce sensitivity • To Maximize Stage Life (Time on Wing) • Rotor life depends on max distress / min life airfoil • Weakest-link structure pervasive in gas turbines • Reducing variation increases rotor life Remove cost from low-impact features Model Inputs

5. Probabilistic Design, Uncertain WorldWhy? To increase the speed of design parametric studies and optimization using engineering model emulators Hours / Run Seconds / Run Maximin Latin Hypercube DOE Output in Design Space Inputs Design Space GEMSA, GPMSA, etc iSight-FD, etc Computer Model • Emulator • Sensitivity • Drive the DOE through the model • Stress • Deflection • Temperature • Life • Performance • Etc. • Structural FEM • CFD model • Matlab code • Fortran code • Other models • Geometric dimensions • Loads • Temperatures • Material properties • Heat transfer coefficients • Etc.

6. DFV Estimated Benefits • Component-level Design For Variation has yielded an estimated 64%-88% return on internal investment. The savings resulted from: • Optimized inspection procedures and tolerances • Reduced quality-related analysis and investigation time • Reduced design iterations • Improved reliability • Improved on-time engine deliveries • Improved root cause investigation process • Based on Six Sigma history and internal trends, the return is expected to increase rapidly in subsequent years • System-level Design For Variation is predicted to yield 40x return on investment due to • Achieving system-level performance and reliability goals earlier in the development cycle • Shorter development programs

7. Design For Variation (DFV) Strategic Plan Vision: All Key Modeling Processes will be DFV-enabled • Strategy • Identify Key Processes • Define elements of a DFV-enabled modeling process • Provide Resources under Strategic Initiative Mechanical Systems and Externals Carbon Seal Performance Ball & Roller Bearing Design FDGS Durability Externals: Forced Response Analysis Combustor and Augmentor Combustor pattern factor Combustor Liner TMF Augmentor Ignition Margin Audit Mid Turbine Frame Robust Design Air Systems Thermal Management Model Internal Air System Model Engine Data Matching DFV Infrastructure (Statistics & Partners) Emulation, Calibration Software High Intensity Computing Parametric Modeling Optimization Training ESW Communications Input Data Tech Support Validation Testing Engine Validation Planning Vehicle Systems Probabilistic Ambient Temp Distribution Performance Analysis Performance Monte Carlo Risk Assessment Engine Test Confidence, Uncertainty Uncertainty in Engine System Predictions Production Test Data Trending and Analysis Statistical Data-match System-Level Risk Communication and Decision Making Fan & Compressor HFB Producibility Parametric Airfoil Compressor Aero Design Compressor Tip Clearances Turbine Turbine Blade Durability Turbine Vanes and BOAS Durability Rotor Thermal Model Airfoil LCF Lifing HSE Combustor / Turbine DFV Structures Probabilistic Rotor Lifing Probabilistic Fracture Mechanics Probabilistic HCF Parametric Geometry Simulation Model Engine Dynamics and Loads

8. 10 Elements of a DFV-Enabled Modeling Process Physics-Based Models • Model Preparation • A robust parametric physics-based model • Model Input Variability and Uncertainty Quantification • Process for retrieving data needed to quantify variability and uncertainty in model inputs • Process for performing statistical analysis/developing statistical model of input data • Preserve correlations • Model Sensitivity Analysis • Process for generating a matrix of space-filling computer experiments (model runs) for emulator development • Process for running the computer code at the space-filling design points • Process for • Building and validating the model emulator • Performing a variance-based sensitivity analysis • Model Calibration • Process for determining what experimental/field data are required for model calibration and measurement uncertainty (amount, characteristics to be measured, ..) • Process for performing Bayesian model calibration: calibrate and bias correct (if needed) and assess residual variation • Uncertainty Analysis • Process for generating a Monte-Carlo sample and driving it through • Parametric model (if fast enough), • Model emulator, or • Bias corrected and calibrated model • Enable Practice • Update local ESW and local training. Put in place a process to ensure the model is capable over time. ONE-TIME PROCESS

9. Design For Variation DEFINE Customer requirements (probabilistic) ANALYZEQuantify model input variation / uncertainty, emulate and calibrate model, perform sensitivity and uncertainty analyses SOLVE Identify ‘optimum’ design that satisfies requirements VERIFY/VALIDATEVariability/Uncertainty model SUSTAIN Stable system of causes of performance variation FiveSteps for Executing a DFV-Enabled Process DEFINE ANALYZE SOLVE VERIFY VALIDATE SUSTAIN

10. How do we define the allowable risk of not meeting a requirement? Risk Requirement DEFINE Design For Variation (DFV): Five Steps Define Customer Requirements Explicit customer requirement Safety Impact: Follow Regulatory Requirements System-Level Job Ticket Metric Impact: Follow flow-down or roll-up process Engine Certification Test Impact • None of the above • Previous acceptable experience or other business considerations • 6 Sigma Criteria • Solve for the probability or rate that minimizes expected total cost

11. Design For Variation ANALYZE Quantify model input variation & uncertainty, emulate & calibrate model, perform sensitivity and uncertainty analyses Model Output • Accounting for uncertainty in • Model input • Model itself Perform Bayesian Model Calibration Run Real World Uncertainty Analysis Refine Distributions of Important Model Inputs Run Experiment Through Engineering Model Design Space Filling Experiment Over Model Input Space Develop Model Emulator, Sensitivity Analysis

12. F w X w y Y Design For Variation ANALYZE : Key Technologies 3. Variance-Based Sensitivity Analysis 1.Latin Hypercube Experimental Designs 4. Bayesian Model Calibration 2. Gaussian Process Emulators

13. Performance characteristic y = f (x1, x2, …, xp) depends on p inputs The variance of y can be approximated by Design For Variation SOLVE Identify optimum design that satisfies requirements • We can reduce by • Reducing :the variance in the inputs x1, x2, …, xp • Reducing :the sensitivity of y to variation in x1, x2, ... , xp SOLVE

14. Design for Variation SOLVE: Robust Design Strategies • Noise Factors • Filter • Isolate • Reduce at source • Inoculate (anneal, heat treat) System • Input Signal • Alter/smooth • Selectively block • Output Response • Calibrate • Average • Control Factors • Robust optimization • Material change • Create multiple operating modes Adapted from: Jugulum, R. and Frey, D. (2007). Toward a taxonomy of concept designs for improved robustness, Journal of Engineering Design, 18:2, 139 - 156 SOLVE

15. VERIFY/VALIDATEincludes Data collection and analysis to validate model input probability distributions Manufacturing process data Material property data Temperatures, pressures, rotor speeds, airflows Flight characteristics (e.g. length, T2 at takeoff, taxi time, ..) Additional calibration of physics-based models Trending in-service parts (wear, performance, etc) where feasible to validate models and their inputs Design For Variation VERIFY/VALIDATE Assumptions made in variability and uncertainty modeling VAL/VER

16. The SUSTAIN phase requires process control to ensure stable and consistent distributions over time Manufacturing Assembly Acceptance Testing Process Certification is vitally important Sustaining capabilities to meet design requirements Identifying production & design improvement opportunities Design Sensitivity and Uncertainty Analyses indicate where process control resources should be focused Design For Variation SUSTAIN Stable system of causes of performance variation SUSTAIN

17. Design For Variation Systematic Process for Designing for and Managing Uncertainty and Variability • Establish probabilistic design requirements • Emulate, calibrate engineering models • Solve for design that meets probabilistic requirements • Look for opportunities for making design less sensitive to variation • Validate and sustain model • Write Engineering Standard Work, develop local training

18. Design For Variation What’s New in 2012? • Additional Training Courses Developed • Automated Multi-physics Workflow • System-Level Design

19. Design For Variation – What’s New? Infrastructure: Enabling Design For Variation • Software • Emulation, Sensitivity Analysis, Model Calibration • Statistical Analysis, Monte Carlo Simulation, Optimization • High Performance Computing Resources • Training • INTRODUCTION • PRACTITIONERS I: SENSITIVITY ANALYSIS, EMULATION, AND DOE • PRACTITIONERS II: ISIGHT-FD FOR SENSITIVITY AND UNCERTAINTY ANALYSIS • PRACTITIONERS III: MODEL CALIBRATION AND UNCERTAINTY ANALYSIS • MANAGERS: INTRODUCTION, REVIEW CHECKLIST • Communication • Wiki, Website, Meetings • Input Data Quality and Availability • Process Capability, Material Properties • Systems Performance, Mission Analysis • Engineering Standard Work

20. Design For Variation - What’s New? • Multi-discipline Automated Workflows • Link disciplines: Aero, Thermal, Structures, Materials, Design • Link components • Enable probabilistic analyses, optimization

21. What’s New - PADME ProgramSystem Level Probabilistic Design & Validation of Engines • PADME is a System-Level Extension of Design For Variation • Quantify uncertainty/risk in system-level metrics • Determine design drivers • Determine optimum path to reduce risk • Design changes • Test changes • PADME Goals • Improve Mature vs. EIS Performance Gap by 33% • Improve Mature vs. EIS Reliability Gap by 33% • Reduce EVP Time by up to 50% PADME: Probabilistic Analysis and Design of Materials and Engines

22. PADME VisionEntire Engine Life Cycle Governed By Uncertainty Quantification and Management Fuel Consumption Delay/Cancellation Rate Weight Cost Rigorously Manage Uncertainty Throughout Life Cycle, Target Validation Testing to Address Largest Sources of Uncertainty

23. PADME: Manage Uncertainty Throughout Engine Life Cycle PADME Governed By System-Level NetworksPopulated By Calibrated Component-Level Emulators Quantification of Uncertainty Enables Optimized Trades on System Level Metrics

24. Uncertainty-Based Design Approach Relies on Calibration of Multivariate Aero-Thermal-Structural Models Using Highly Instrumented Engine PADME Strategy Deterministic Design Deterministic Redesign Deterministic Redesign Engine Test Engine Test Engine Test crack Legacy Approach oxidation Engine Endurance Test Robust Design Probabilistic Design R&D Rig/Engine Test DFV- PADME Approach EAR Export Classification: EAR 99

25. Design For Variation – For More Information • Statistical Engineering Issue

26. True Process Value Model Prediction Design Criteria Design For Variation • Goal: quantify, understand, and control the risk of not meeting design criteria or exceeding thresholds • “The revolutionary idea that defines the boundary between modern times and the past is the mastery of risk: the notion that the future is more than a whim of the gods and that men and women are not passive before nature.” • Peter Bernstein, “Against the Gods: The remarkable story of risk”