ENM 310 Design of Experiments and Regression Analysis Chapter 1

1. ENM 310Design of Experiments and Regression AnalysisChapter 1 Ilgın ACAR Spring 2019

3. Introduction to DOE • An experimentis a test or series of tests. • DOE begins with determining the objectivesof an experiment and selecting the process factors for the study. • An Experimental Design is the laying out of a detailed experimental plan in advance of doing the experiment. • Experiments are used widely in the engineering studies • Process characterization and optimization • Evaluation of material properties • Product design and development • Component and system tolerance determination

4. Planning, Conducting and Analyzing and Experiment • Statement of the problem • Choice of the factors, levels and ranges • Selection of the response variables • Choice of design • Conducting the experiment randomly • Statistical analysis • Drawing conclusions and recommendations

5. Design of Experiments

6. Response, factor and model? A response is a measurableresult. • yield of a chemical reaction (chemicalprocess) • deposition rate(semiconductor) • gas mileage (automotive) Response

7. Response, factor and model? A factoris any variable that you think may affect a response of interest. We begin by considering three types of factors – continuous, categorical and blocking continuous factors take any value on aninterval e.g. octane rating [8993] categorical factors have a discrete number of levels e.g. brand [BP, Shell, Exxon] blocking factors are categorical but not generallyreproducible e.g. driver to driver variability

8. Response, factor and model? Factor(s) Model Response(s) a simplified mathematical surrogate for theprocess Examples of models: Comparing three brands of gasoline using an ANOVAmodel: Finding the effect of octane rating using a regressionmodel: Y (the response) is the mileage of a car in miles pergallon.

9. ANOVA and regression models areequivalent… Replace with 0 and 1 and 2 with 1 and2.

10. Important Points from the Fathers ofDOE DOE – Problem solving methodology for efficiently identifying cause-and-effectrelationships. Fisher’s Four Fundamentals ofDOE Factorialprinciple Randomization Blocking Replication R.A.Fisher “To discover what happens to a process when a factor is changed, you must actually changeit!” GeorgeBox

11. Factorial Experiment Design • In a factorialexperimentdesign, allpossiblecombinations of factorlevelsartetested.

12. Factorial experiment design Effect of a factor: The average change in response when a factor's level is increased from small to large.

13. Interaction effect

14. Golf example • The type of driver used (oversized or regular sized) • The type of ball used (balata or three piece) • Walking and carrying the golf clubs or riding in a golf cart • Drinking water or drinking “something else” while playing • Playing in the morning or playing in the afternoon • Playing when it is cool or playing when it is hot • The type of golf shoe spike worn (metal or soft) • Playing on a windy day or playing on a calm day.

16. Overview The first question you have to ask yourself is what you want to do. Do you want to • Compare two or more groups, or one group to a fixed value? • Hypothesis test • Screen the observed responses to identify factors/effects that are important? • Screening investigation • Maximize or minimize a response (variability, distinct to target, robustness)? • Optimization problem • Develop a regression model to quantify the dependence of a response variable on the process input? • Statistical Modeling

17. Design and Analysis of Single-Factor Experiments: The Analysis of Variance Suppose that a civil engineer is investigating the effects of different curing methods on the mean compressive strength of concrete. The experiment would consist of making up several test specimens of concrete using each of the proposed curing methods and then testing the compressive strength of each specimen. The data from this experiment could be used to determine which curing method should be used to provide maximum mean compressive strength. • Could be designed and analyzed using the statistical hypothesis methods for two samples The civil engineer may want to investigate five different curing methods. • The ANalysis Of VAriance(frequently abbreviated ANOVA) can be used for comparing means when there are more than two levels of a single factor.

18. Design and Analysis of Single-Factor Experiments: The Analysis of Variance ANOVA 1: Completely Randomized Design Suppose that we want to compare the fuel consumption recorded for three different makes of cars over a specified distance. Let’s assume that 18 drivers are chosen: six are randomly assigned to A-cars, six to B-cars, and six to C-cars. Test: H0: The mean fuel consumption of the three makes of cars is the same. H1: At least two car makes have different mean fuel consumptions. Assume that the sample means resulting from our test are as follows: Sample mean for A-cars = 20 mpg   Sample means for B-cars = 21 mpg   Sample means for C-cars = 22 mpg  Are the cars different, with respect to mpg?

19. ANOVA • A manufacturer of paper used for making grocery bags is interested in improving the product’s tensile strength. Product engineering believes that tensile strength is a function of the hardwood concentration in the pulp and that the range of hardwood concentrations of practical interest is between 5 and 20%. A team of engineers responsible for the study decides to investigate four levels of hardwood concentration: 5%, 10%, 15%, and 20%. They decide to make up six test specimens at each concentration level by using a pilot plant. All 24 specimens are tested on a laboratory tensile tester in random order. The data from this experiment are shown in Table 13-1.

20. The Analysis of Variance • The levels of the factor are sometimes called treatments, • Each treatment has nobservationsor replicates. • The responsefor each of the a treatments is a random variable. • The role of randomizationin this experiment is extremely important.

21. The Analysis of Variance • a levels (treatments) of a factor and n replicates for each level. • yij: the jth observation taken under factor level or treatment i.

22. Models for the Data • Means model: • yij is the ijthobservation, • i is the mean of the ith factor level • ij is a random error with mean zero • Effects model: • yijis a random variable denoting the (ij)th observation, • μis a parameter common to all treatments called the overall mean, • τiis a parameter associated with the ith treatment called the ithtreatment effect, • eijis a random error component.

23. Linear statistical model • One-way or Single-factor analysis of variance model • Completely randomized design: the experiments are performed in random order so that the environment in which the treatment are applied is as uniform as possible. • For hypothesis testing, the model errors are assumed to be normally and independently distributed random variables with mean zero and variance, • Fixed effect model: a levels have been specifically chosen by the experimenter.

24. Fixed-effect vs Random-effects The a factor levels in the experiment could have been chosen in two different ways. • The experimenter could have specifically chosen the a treatments. • to test hypotheses about the treatment means, and conclusions cannot be extended to similar treatments that were not considered. • to estimate the treatment effects. This is called the fixed-effects model. • The a treatments could be a random sample from a larger population of treatments. In this situation, we would like to be able to extend the conclusions (which are based on the sample of treatments) to all treatments in the population whether or not they were explicitly considered in the experiment. • The treatment effects τiare random variables, and knowledge about the particular ones investigated is relatively unimportant. • test hypotheses about the variability of the τiand try to estimate this variability. This is called the random-effects, or components of variance model.

25. Analysis of the Fixed Effects Model • yi. represents the total of the observations under the ithtreatment. • Let i. represent the average of the observations under the ithtreatment. • Similarly, let y.. represent the grand total of all the observations and .. represent the grand average of all the observations. Expressed symbolically, We are interested in testing the equality of the a treatment means that is The appropriate hypotheses are:

26. Analysis of the Fixed Effects Model • to write the above hypotheses is in terms of the treatment effects 𝜏i, say • testing the equality of treatment means or testing that the treatment effects (the 𝜏i) are zero. • The appropriate procedure for testing the equality of a treatment means is the analysis of variance.

27. Decomposition of the Total Sum of Squares • The name analysis of variance is derived from a partitioning of total variability into its component parts. The total corrected sum of squares SSTreatments = Sum of Squares for Treatments (variability between or among) SSE =Sum of Squares for Error (variability within) SST = Total Sum of Squares an= N total observations

28. Terms: If each of these sums of squares is divided by its associated degrees of freedom, then we have the respective mean squares(sample variance of the y’s.) That is: where:   degrees of freedom associated with SSTreatment = a – 1  degrees of freedom associated with SSE = N-a

29. A large value of SSTreatmentsreflects large differences in treatment means • A small value of SSTreatments likely indicates no differences in treatment means • dfTotal = dfTreatment + dfError • If there are no real differences among the n group means, then the three mean squares (above) provide estimates of 2 • No differences between a treatment means: variance cane be estimated by

30. Statistical Analysis • H0∶𝜇1 = 𝜇2 = · · · = 𝜇a,= 0 or equivalently, • H0∶𝜏1 = 𝜏2 = · · · = 𝜏a= 0 • Test statistics • is distributed as F with a − 1 and N − a degrees of freedom • Reject H0 if

31. Rewrite the equation

32. The analysis of variance (or ANOVA) table

33. Example 2 • An engineer is interested in investigating the relationship between the RF power setting and the etch rate for this tool. The objective of an experiment like this is to model the relationship between etch rate and RF power and to specify the power setting that will give a desired target etch rate. She is interested in a particular gas C2F6(hexafluoroethane), and gap (0.80 cm) and wants to test four levels of RF power: 160, 180, 200, and 220 W. She decided to test five wafers at each level of RF power. • the engineer is interested in determining if the RF power setting affects the etch rate, and she has run a completely randomized experiment with four levels of RF power and five replicates.

34. ANOVA Example – Completely Randomized Design The tensile strength of Portland cement is being studied. Four different mixing techniques can be used economically. The following data have been collected (randomly!) Test the hypothesis that mixing techniques effect the strength of the cement at a 0.05 significance level ( = 0.05)   H0: T1 = T2 = T3 = T4   H1: At least two means are different

35. ANOVA Example Resulting ANOVA table: Note: F0 = 12.73, and from the F-table we find that: F0.05, 3, 12 = 3.49 we reject the null hypothesis (since F0 > F0.05, 3, 12) and conclude that there is evidence of a significant difference in the average tensile strength of cement between the four mixing techniques. That is, at least one mixing technique mean is statistically different from the others.

36. Estimation of the Model Parameters • Model: yij = μ + τ i + ε ij • Estimators: • Confidence intervals: a 100(1 − 𝛼) percent confidence interval on the ith treatment mean 𝜇i A 100(1 − 𝛼) percent confidence interval on the difference in any two treatment means, say 𝜇i− 𝜇j,

37. Unbalanced Data • Letni observations be taken under treatment i, i=1,2,…,a, N = ini , There are two advantages in choosing a balanced design. The test statistic is relatively insensitive to small departures from the assumption of equal variances for the a treatments if the sample sizes are equal. This is not the case for unequal sample sizes. 2. The power of the test is maximized if the samples are of equal size. z

38. ANOVA Assumptions To appropriately use the ANOVA procedure, the following assumptions must be true. 1. Randomness and independence. (Random and independent samples). MUST BE TRUE! 2. Normality. Each population from which the samples are drawn from must be normally distributed. Robust to this assumption, especially with large samples. 3. Homogeneity of Variance. The populations from which the samples are drawn from must have equal variances. Robust to this assumption when sample sizes are equal.

39. Model Adequacy Checking • Assumptions: yij ~ N(μ + τ i, σ2) • The examination of residuals • Definition of residual: • If the model is adequate, the residuals should be structureless.(Contain no obvious pattern)

40. The Normality Assumption • Plot a histogram of the residuals • Plot a normal probability plot of the residuals • See Table 3-6

41. May be • the left tail of error is thinner than the tail part of standard normal • Outliers • The possible causes of outliers: calculations, data coding, copy error,…. • Sometimes outliers are more informative than the rest of the data.

42. Detect outliers: Examine the standardized residuals, If the errors 𝜖ijare N(0,𝜎2), the standardized residuals should be approximately normal with mean zero and unit variance. Thus, • about 68 percent of the standardized residuals should fall within the limits ±1, • about 95 percent of them should fall within ±2, • all of them should fall within ±3. • a residual bigger than 3 or 4 standard deviations from zero is a potential outlier.

43. Independence Assumption Plot of Residuals in Time Sequence • Plotting the residuals in time order of data collection is helpful in detecting correlation between the residuals. • Residuals should fall within ±3randomly. • If there is a dependency, this is a serious problem. it is important to prevent the problem if possible when the data are collected. • Dots should not show any pattern.

45. Homogeneity of Variances Assumption Constant Variance Test • Determining whether the distribution of errors varies according to the factor levels. • Nonconstant variance: the variance of the observations increases as the magnitude of the observation increase, i.e. yij 2 • If the factor levels having the larger variance also have small sample sizes, the actual type I error rate is larger than anticipated. • If the factor is decreasing / increasing according to the levels, there is «funnel effect». • In case of funnel effect in the distribution of errors, this effect is tried to be eliminated by transformation techniques. Variance-stabilizing transformation

46. Statistical Tests for Equality Variance: • Bartlett’s test: • Reject null hypothesis if

47. Comparing Pairs of Treatment Means : Tukey Test • The ANOVA test is used as an overall test of the difference of means; however, it does not identify which means are statistically different. • Suppose that, following an ANOVA in which we have rejected the null hypothesis of equal treatment means, we wish to test all pairwise mean comparisons: H0∶𝜇i=𝜇j H1∶𝜇i≠ 𝜇j • Tukey’s procedure makes use of the distribution of the studentized range statistic

48. Procedure Step 1:Obtain the critical range for a given significance level, : • For equal sample sizes, Tukey’s test declares two means significantly different if the absolute value of their sample differences exceeds • When sample sizes are not equal (Tukey–Kramer procedure) • Step 2: construct a set of 100(1 − 𝛼) percent confidence intervals for all pairs of means

49. Return to ANOVA example:  Since the ANOVA F-test was significant, it is appropriate to apply the Tukey Test. Step 1: Only one critical range is necessary, since the sample size for the cement samples are equal. Assume  =0.05. a = 4 and n – a = 12 qu = 4.20 (from Table)  MSE = 12,826

50. ANOVA 2: Randomized Block Design Suppose that we want to compare the fuel economy obtained by three makes of cars, , , and . We will consider an experiment in which six trials are to be run with each type of car. If these trials are conducted using six drivers, each of whom drives a car of all three types, it will be possible to extract from the results information about driver variability as well as information about differences among the three types of cars, since every car make will have been tested by every driver. The cars are the treatment variables; in this case there are three treatments. The additional variable in this case is the driver, which is called the blocking variable. The experiment is said to be arranged in blocks, in our example, there are six blocks, one for each driver. This kind of block design allows us to obtain information about two factors simultaneously; however, it restricts randomization. Why is it called a Randomized Block Design? The drivers (blocks) are randomized within each treatment when collecting the sample.