Design and Analysis of Experiments

Design and Analysis of Experiments Dr. Tai-Yue Wang Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC

Analysis of Variance Dr. Tai-Yue Wang Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC

Outline(1/2) • Example • The ANOVA • Analysis of Fixed effects Model • Model adequacy Checking • Practical Interpretation of results • Determining Sample Size

Outline (2/2) • Discovering Dispersion Effects • Nonparametric Methods in the ANOVA

What If There Are More Than Two Factor Levels? • The t-test does not directly apply • There are lots of practical situations where there are either more than two levels of interest, or there are several factors of simultaneous interest • The ANalysis Of VAriance(ANOVA) is the appropriate analysis “engine” for these types of experiments • The ANOVA was developed by Fisher in the early 1920s, and initially applied to agricultural experiments • Used extensively today for industrial experiments

An Example(1/6)

An Example(2/6) • An engineer is interested in investigating the relationship between the RF power setting and the etch rate for this tool. The objective of an experiment like this is to model the relationship between etch rate and RF power, and to specify the power setting that will give a desired target etch rate. • The response variable is etch rate.

An Example(3/6) • She is interested in a particular gas (C2F6) and gap (0.80 cm), and wants to test four levels of RF power: 160W, 180W, 200W, and 220W. She decided to test five wafers at each level of RF power. • The experimenter chooses 4 levels of RF power 160W, 180W, 200W, and 220W • The experiment is replicated 5 times – runs made in random order

An Example --Data

An Example – Data Plot

An Example--Questions • Does changing the power change the mean etch rate? • Is there an optimum level for power? • We would like to have an objective way to answer these questions • The t-test really doesn’t apply here – more than two factor levels

The Analysis of Variance • In general, there will be alevels of the factor, or atreatments, and nreplicatesof the experiment, run in randomorder…a completely randomized design (CRD) • N = an total runs • We consider the fixed effectscase…the random effectscase will be discussed later • Objective is to test hypotheses about the equality of the a treatment means

The Analysis of Variance

The Analysis of Variance • The name “analysis of variance” stems from a partitioning of the total variability in the response variable into components that are consistent with a model for the experiment

The Analysis of Variance • The basic single-factor ANOVA model is

Models for the Data • There are several ways to write a model for the data • Mean model • Also known as one-way or single-factor ANOVA

Models for the Data • Fixed or random factor? • The a treatments could have been specifically chosen by the experimenter. In this case, the results may apply only to the levels considered in the analysis. fixed effect models

Models for the Data • The a treatments could be a random sample from a larger population of treatments. In this case, we should be able to extend the conclusion to all treatments in the population.  random effect models

Analysis of the Fixed Effects Model • Recall the single-factor ANOVA for the fixed effect model • Define

Analysis of the Fixed Effects Model • Hypothesis  

Analysis of the Fixed Effects Model • Thus, the equivalent Hypothesis

Analysis of the Fixed Effects Model-Decomposition • Total variability is measured by the total sum of squares: • The basic ANOVA partitioning is:

Analysis of the Fixed Effects Model-Decomposition • In detail (=0)

Analysis of the Fixed Effects Model-Decomposition • Thus SST SSTreatments SSE

Analysis of the Fixed Effects Model-Decomposition • A large value of SSTreatmentsreflects large differences in treatment means • A small value of SSTreatments likely indicates no differences in treatment means • Formal statistical hypotheses are:

Analysis of the Fixed Effects Model-Decomposition • For SSE • Recall

Analysis of the Fixed Effects Model-Decomposition • Combine a sample variances • The above formula is a pooled estimate of the common variance with each a treatment.

Analysis of the Fixed Effects Model-Mean Squares • Define and df df

Analysis of the Fixed Effects Model-Mean Squares • By mathematics, • That is, MSE estimates σ2. • If there are no differences in treatments means, MSTreatments also estimatesσ2.

Analysis of the Fixed Effects Model-Statistical Analysis • Cochran’s Theorem Let Zi be NID(0,1) for i=1,2,…,v and then Q1, q2,…,Qs are independent chi-square random variables withv1, v2, …,vs degrees of freedom, respectively, If and only if

Analysis of the Fixed Effects Model-Statistical Analysis • Cochran’s Theoremimplies that are independently distributed chi-square random variables • Thus, if the null hypothesis is true, the ratio is distributed as F with a-1 and N-a degrees of freedom.

Analysis of the Fixed Effects Model-Statistical Analysis • Cochran’s Theoremimplies that Are independently distributed chi-square random variables • Thus, if the null hypothesis is true, the ratio is distributed as F with a-1 and N-a degrees of freedom.

Analysis of the Fixed Effects Models-- Summary Table • The reference distributionfor F0 is the Fa-1, a(n-1) distribution • Reject the null hypothesis (equal treatment means) if

Analysis of the Fixed Effects Models-- Example • Recall the example of etch rate, • Hypothesis

Analysis of the Fixed Effects Models-- Example • ANOVA table

Analysis of the Fixed Effects Models-- Example • Rejection region

Analysis of the Fixed Effects Models-- Example P-value • P-value

Analysis of the Fixed Effects Models-- Example One-way ANOVA: Etch Rate versus Power Source DF SS MS F P Power 3 66871 22290 66.80 0.000 Error 16 5339 334 Total 19 72210 S = 18.27 R-Sq = 92.61% R-Sq(adj) = 91.22% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev ---+---------+---------+---------+------ 160 5 551.20 20.02 (--*---) 180 5 587.40 16.74 (--*---) 200 5 625.40 20.53 (--*---) 220 5 707.00 15.25 (--*---) ---+---------+---------+---------+------ 550 600 650 700 Pooled StDev = 18.27 • Minitab

Analysis of the Fixed Effects Model-Statistical Analysis • Coding the observations will not change the results • Without the assumption of randomization, the ANOVA F test can be viewed as approximation to the randomization test.

Analysis of the Fixed Effects Model-Estimation of the model parameters • Reasonable estimates of the overall mean and the treatment effects for the single-factor model are given by

Analysis of the Fixed Effects Model-Estimation of the model parameters • Confidence interval for μi • Confidence interval for μi- μj

Analysis of the Fixed Effects Model- Unbalanced data • For the unbalanced data

A little (very little) humor…

Model Adequacy Checking • Assumptions on the model • Errors are normally distributed and independently distributed with mean zero and constant but unknown variances σ2 • Define residual where is an estimate of yij

Model Adequacy Checking--Normality • Normal probability plot

Model Adequacy Checking--Normality • Four-in-one

Model Adequacy Checking--Plot of residuals in time sequence • Residuals vs run order

Model Adequacy Checking--Residuals vs fitted • Residuals vs fitted

Model Adequacy Checking--Residuals vs fitted • Defects • Horn shape • Moon type • Test for equal variances • Bartlett’s test

Model Adequacy Checking--Residuals vs fitted • Test for equal variances • Bartlett’s test Test for Equal Variances: Etch Rate versus Power 95% Bonferroni confidence intervals for standard deviations Power N Lower StDev Upper 160 5 10.5675 20.0175 83.0477 180 5 8.8384 16.7422 69.4591 200 5 10.8357 20.5256 85.1557 220 5 8.0496 15.2480 63.2600 Bartlett's Test (Normal Distribution) Test statistic = 0.43, p-value = 0.933 Levene's Test (Any Continuous Distribution) Test statistic = 0.20, p-value = 0.898

Design and Analysis of Experiments