1 / 0

Part 5

Analysis. Part 5. Course Outline. Day 1 Day 2. Part 0: Student Introduction Paper Helicopter - Pt 0 Use what you know Part 1: DOE Introduction What is a Designed Experiment? Part 2: Planning Understand the test item’s process from start to finish

archer
Download Presentation

Part 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis

    Part 5

  2. Course Outline Day 1 Day 2 Part 0: Student Introduction Paper Helicopter - Pt 0 Use what you know Part 1: DOE Introduction What is a Designed Experiment? Part 2: Planning Understand the test item’s process from start to finish Identify Test Objective – screen, characterize, optimize, compare Response variables Identify key factors affecting performance Paper Helicopter - Pt 1 Planning Part 3: Hypothesis Testing Random variables Understanding hypothesis testing Demonstrate a hypothesis test Sample size, risk, and constraints Seatwork Exercise 1 – Hang time Measurements Part 4: Design and Execution Part 4: Design and Execution Understanding a test matrix Choose the test space – set levels Factorials and fractional factorials Execution – randomization and blocking In-Class F-18 LEX Study - Design Build Paper Helicopter - Pt 2 Design for Power Part 5: Analysis Regression model building ANOVA Interpreting results – assess results, redesign and plan further tests Optimization In-Class F-18 LEX Study - Analysis Seatwork Exercise 2 – NASCAR Paper Helicopter – Pt 3 Execute and Analyze Paper Helicopter – Pt 4 Multiple Response Optimization Part 6: Case Studies
  3. Science of Test IV Metrics of Note Plan Sequentially for Discovery Factors, Responses and Levels Design with Confidence and Power to Span the Battlespace N, a, Power, Test Matrices Analyze Statistically to Model Performance Model, Predictions, Bounds DOE Execute to Control Uncertainty Randomize, Block, Replicate
  4. General regression model We seek to develop an input–output relationship between factors (x) and responses (Y), write as Y=f(x) But, experimental processes are subject to random error so our general model with error becomes Y=f(x)+e We can now write out our relationship in a general regression model form:
  5. First order regression model y b0 b1 x1 The b’s in the model are the regression coefficients, unknown before the experiment Lets look at a first order model with one factor: Easily recognizable as the equation for a line, where b0 is the intercept and b1is the slope, Y is the dependent variable (response) and x1 the independent variable (factor) Expanding to more factors (multivariate) but still building a 1st order model:
  6. REBEL AGM (3 Factors: Weapon, Range, Altitude) Real World Matrix is Coded: All High Values of xi’s = +1 All Low Values of xi’s = -1

    Coding Example

    R E S P O N S E R E S P O N S E
  7. Coding The regression model is best suited to coded factors This allows us to compare relative magnitudes (importance) of regression coefficients later We write the x’s as + 1 or –1 Let’s revisit the REBEL missile comparison We will build a first order plus interaction regression model now This is the typical starting point in a DOE, a factorial design gives: Intercept 3 Two Factor Interactions 3 Main Effects Error 3 F.I.
  8. Introduction 22 factorial design matrix Each factor of a 2k factorial design is chosen at two levels The high and low level are denoted as (+) and (-) in a table of signs known as contrast constants Signs are arranged in standard order (recall that column in DX7) The interaction column is a product of the signs of the component columns eg. AB=A*B Pairs of signs Product of1st 2 columns Start here, alternate signs
  9. Finding Effects Using the treatment combinations, the effects contributions may be found easily – simply add and subtract the responses with signs given by the column
  10. Example 22 Design Consider a sensor assessment study considering two factors that may impact the target location error  Look down angle  Slant Range  
  11. Analysis – Signal: Grand Mean or Intercept TLE radial miss data 22 10 y 22, 25, 19 5, 15, 10  B: Angle  A: Range   35, 37, 36 56, 51, 61 36 56 We’re going to solve four equations for four unknowns
  12. Analysis – Signal: Factor Effects TLE radial miss data 22 10 22 10 y 22, 25, 19 5, 15, 10 36 56  22 22 10 10 B: Angle 36 36 56 56  A: Range   35, 37, 36 56, 51, 61 36 56
  13. Analysis - Noise TLE radial miss data 9 25 9 25 y 22, 25, 19 5, 15, 10  1 25 B: Angle  A: Range   35, 37, 36 56, 51, 61 1 25
  14. Analysis – Signal to Noise *Significant if large S/N ratio, depends on N, but reasonable ROT >3
  15. Regression Model General Model Regression Model 28 LB 38 48 UB Response Surface
  16. We solved the system of equations for the β’s while minimizing the error and got the result below, the range coefficient was very small compared to weapon or alt What conclusions can we make from this graph?

    Regression Analysis – Model Interpretation

    Data points Error Miss Distance (ft) 5 10 15 Altitude (x1000 ft) 10 20 Surface described by regression equation vA vB Weapon Miss distance increases from vA to vB: β1 will be positive and large compared to error (e) Miss distance increases with Altitude, β2 will be positive and large compared to error (e)
  17. Interpreting Results and Further Testing Once you’ve made conclusions on your initial data…then what? Test -> Analyze -> Test Reserve some resources for re-testing based on what you’ve learned Give enough time between test events to analyze data Choose some random points for model validation Report conclusions with confidence and power

  18. Example 1 Analysis:Maverick AGM

  19. Stage III: Execute Flight Test
  20. Stage IV – Analyze Data Compute effect estimates Building estimating equation Regression model using least squares: f(effect estimates, error)
  21. Miss Distance Performance
  22. Interaction: Range and Airspeed
  23. How Factorial Matrices Work -- a peek at the linear algebra under hood We set the X settings, observe Y Solve for b s.t. the error (e) is minimized Simple 2-level, 2 X-factor design Where y is the grand mean, A,B are effects of variables alone and AB measures variables working together
  24. How Factorial Matrices Work II To solve for the unknown b’s, X must be invertible Factorial design matrices are generally orthogonal, and therefore invertible by design
  25. Maverick AGM Performance ResultsMiss Distance Estimating Equation Miss Distance = 17.80 +7.16 * Airspeed -7.58 * Range +8.28 * Airspeed * Range Model - Coded Units (Low=-1, High=+1) Estimation
  26. Sequential Assembly Using a building block analogy We start with the foundation – a factorial experiment We check themodel for fit If needed, weadd more pointsto build higherorder model C B A Add axial points (red) CCD Supports 2nd order model Factorial with Centers (green) First order model with 2FI’s
  27. DOE Process Recap I Plan the Project 1. Statement of the problem -- why this, now? 2. Objective of the experiment -- screen, characterize, optimize, compare? 3. Response variables - the process outputs via Process Flow 4. Potential causal variables – brainstorm physical variables – via CNX diagram. II Design the Test Matrix 5. Constraints -- duration, analysis cycle time, events/sortie 6. Prioritize factors -- Control, Fixed, Noise 7. Select statistical design – Factorial, 2k, etc. 8. Write test plan with sample matrices, data and sample output III Execute the Test 9. Randomize run order and block as needed 10. Execute the control factors and record anomalies IV Analyze the Data and Recommend 11. Acquire, reduce, look, explore, analyze and project data 12. Draw conclusions, redesign, assess results and plan further tests
  28. In-Class Example: F-18 LEX Study Follow along with DX 7 File Goals Demonstrate analysis in DX 7 ANOVA table – determine significant terms Regression model for each response Model Interpretation Start with the interaction graphs Review other main effects Diagnostics – we have not emphasized but is important Demonstrate normality, constant variance, and linear independence checks
  29. F-18 LEX Study – Review Test Objectives Wind Tunnel Test DOE Question: is the new LEX better than the old LEX ? Responses Factors PROCESS: Lift Coefficient desire increase LEX Type (Old or New) Drag Coefficient desire no change F-18 Aerodynamics Angle of Attack (range) Pitch Moment Coefficient desire no change Angle of Sideslip (range) Stabilizer Deflection (range) Stabilizer deflection is trailing edge down positive
  30. Analysis with Design Expert Open the file: F-18 LEX Study Analysis.dx7 Click on Analysis, Lift, Effects, plot of effects Click on ANOVA, note significant effects While D (LEX type is not significant – Interaction BD is) Scroll down to the regression model Now go to Model Graphs Under the factors tool pick the term BD This is the Sideslip x LEX Type interaction (adjust y axis – right click – preferences, y axis 0.3 -0.42) Note the superior performance of D1 vs. D2 at low sideslip angles Adjust other factors to see effect on lift Conclusions – LEX D1 does afford more lift at low sideslip angles (0-2), no difference beyond about 2 deg
  31. Analysis with Design Expert Now lets look at Drag, Click on Analysis, Drag, Effects, plot of effects Click on ANOVA, note significant effects Angle of Attack, Stabilizer deflection and their interaction No LEX effect, No sideslip effect Scroll down to the regression model Now go to Model Graphs Under the factors tool pick the term AC This is the Angle of Attack x Stabilizer interaction Stabilizer has little effect on drag at low angle of attack but at high angle of attack there is an increase in drag at positive deflections of the stabilizer Conclusions: the choice of LEX has no effect on drag
  32. Analysis with Design Expert Now lets look at Pitching Moment, Click on Analysis, Pitching Moment, Effects, plot of effects Click on ANOVA, note significant effects Stabilizer deflection is the only significant effect LEX choice does not change pitching moment response Scroll down to the regression model Now go to Model Graphs Under the factors tool pick the term C This is the Stabilizer main effect Deflecting the stabilizer down (+5 deflection) gives a nose down pitching moment (negative) Conclusions: the choice of LEX has no effect on pitching moment
  33. Analysis with Design Expert Design Expert provides so much more Diagnostics Check the assumptions of normally distributed, independent, random error with constant variance – look at residuals, ei In the ANOVA table Estimates of uncertainty – look at std dev Check for lack of fit and quadratic curvature Other summary stats In the Design, Evaluation section Power estimates to detect effects with a delta in std. dev.
  34. Analysis Linear algebra and factorial matrix analysis Regression Equation ANOVA Interpreting Results Use of Design Expert to do the heavy lifting

    Session 5: Summary

  35. Seatwork Do Seatwork Exercise 2 – NASCAR
  36. Seat Work Exercise 2 Perform an analysis on a NASCAR wind tunnel full factorial design experiment 1997 NASCAR Winston Cup Monte Carlo Race Car ODU LFST (full-scale wind tunnel) 4 Factors Front ride height Rear ride height Yaw angle % Grille tape 3 Responses Front Downforce Rear Downforce Drag Rear Downforce Front Downforce Drag Yaw Grille Tape
  37. Full-Factorial Design Full factorial 24 16 runs + + 4 centers = 20 runs total Randomized test matrix One block Follow the ex.Instructions andanswer the questions
  38. Paper Helicopter Design Do Paper Helicopter Design – Pt 3 Execute and Analyze Do Paper Helicopter Design – Pt 4 Multiple Response Optimization
More Related