Experimental Design

1. Experimental Design Planning experiments to allow collection of valid data & powerful statistical analysis of the data Objective: Professor Zewei Luo, Office: S224, Tel: 45404 E-mail: z.luo@bham.ac.uk

2. Example I. Does gastric freezing relieve gastric ulcers? • The design I: • Patients with gastric ulcers swallow deflated balloon into which a • refrigerated liquid was introduced • Data published in Journal of American Medical Association • showed that patients so treated had less stomach acid and pain • than untreated patients • The design II: • Patients with gastric ulcers are divided into two groups. • Patients in group I are given balloons containing refrigerated • liquid and those in group II are given balloons containing nothing • (placebo). None of the patients are told which balloon are given! • Data based on this experiment showed 34% improved with cold • treatment but 34% improvement was also observed with placebo! An appropriate control is needed to make a sensible comparison!

3. Example II. Do cats prefer Mousey (M) to Doggy (D)? • The design: • Data: • two cat foods: Mousey (M) & Doggy (D) • to measure amount of food eaten in grams when offered to a cat. • to give M and D to n cats each. So we need 2n cats (n  2). • we have to group the 2n cats randomly into the two feeding groups M group: x1, x2, ….,xn D group: y1, y2, ….,yn

4. It can be seen that to calculate and , n must be  2 ! i.e. each of experimental treatments must be applied to more than one subjects (replication)! Test for difference in preference between the two cat foods is equivalent to test significance between the two sample means by, for example, t - test

5. Then the two means are The expected value of the t statistic is now even when m=d Consequences when (i). the cats are not randomly grouped; For example, n fat cats are fed with Mousey and the rest are fed with Doggy that is, i.e. the statistical test may be biased without incorporating randomization mechanism in the experimental design!

6. whenever m=d However, if the cats are randomly grouped Where di = 1 if cat i is fat and 0 otherwise; li = 1 if cat i is slim and 0 otherwise and Pr{di = 1} = Pr{di = 0} = Pr{li = 1} = Pr{li = 0} =1/2, i.e. fat or slim cats are equally likely given “Mousey” or “Doggy”, then the expected value of the t statistic is now Thus, with randomization, background of experimental treatments can be homogenized and thus systematic errors can be effectively removed!

7. (ii). influences of replication in an experimental design (explained by the cat food example) • increasing replicate number increases accuracy of mean estimate because • increasing replication increases d.f. of t test statistic (2n-2) and in turn • decreases the threshold to disprove Ho, for example A modest number of replicates lowers the “hurdle” for the test statistic to jump over to reach significance

8. For example, one plans to test 10% difference between the two cat food with = 25 and • How many replicates are needed to detect a genuine difference in • preference between the two cat foods with a given confidence of 1-a? You should know (a). mx = E(x), my = E(y) and (b). s2 = Var(ei) They can be obtained from published data or estimated from a trial. The above inequality equation can be solved for any given a. For example, when a =0.05, n  16

9. Gradient in fertility of soil High Low Small area has a homogeneous environment so small s2 Large area has a heterogeneous environment so s2 large Restraints on scale of experiments? Why experimental scale matters? • balance between costs of a large scale and gains in experimental • efficiency • large scale may cause un-control of experimental errors, for example A field trial to test yield performance of a set of varieties

10. Gradientin fertility of soil High Low Solution: divide experiment into several sub-experiments “blocks” and dissect variation of block component from total variation

11. Gradient in fertility of soil High Low Yes No Randomized blocks Block at right angles to gradient if known

12. Design of randomised blocks • Each block would contain the same number of subjects (plants). • Each block would contain all the treatments. • The plants and treatments in each block would be separately randomised. • So, each block is a complete but small version of the experiment. • The environmental effect of the gradient is therefore removed and s2 minimised.

13. Gradient Completely randomised 5 randomised blocks The example

14. Item df SS MS F Prob. Between varieties 1 174.05 174.05 4.67 0.05-0.01 Within 18 670.50 37.25 varieties 19 844.55 Total One Way ANOVA

15. ANOVA table

16. Effect of blocking • The replicate error within varieties is very much reduced and so the ‘Variety’ effects may be detected with larger chance (experimental power) • Increases Power of test • Called a Randomised Complete Block Expt. (RCB).

17. Types of ‘block’ • Shelves in incubator • Different occasions or sites • Different operators

18. The “1C + 3Rs” principles of experimental designs • need of “positive” and/or “negative” control to contrast genuine effects of experimental treatments (C) • need of randomization to homogenize background of experimental treatment(s) and to remove systematic errors (R) • need of replication to increase experimental power (R) • need of restraint on scale of experimental units (blocks) to dissect un-controllable variation from total variation and, in turn, to improve experimental power (R).