Maximizing Experimental Design for Accurate Results

1. Section 5.2 EXPERIMENTAL DESIGN

2. EXPERIMENTAL UNITS, SUBJECTS AND TREATMENTS • Experimental Unit – The individuals on which the experiment is being conducted • Subjects – Experimental Units that are human beings • Treatment – A specific experimental condition that is applied to the units – in an attempt to see if it has an effect

3. EXPERIMENTAL UNITS, SUBJECTS AND TREATMENTS - EXAMPLES • Experimental Unit – 1 acre of land … to have corn planted and harvested later • Subjects – A person who will take a test of some sort • Treatment – For the above, different brands of fertilizer used… or for the subjects, you might have different types of teaching methods, or background music

4. FACTORS and LEVELS • Factor: The Explanatory Variable … For example … A quantity of fertilizer, or drug, or volume of music • Level: The amount of the factor being applied to the experimental unit … for example … 100 lb, 200 lb, 500lb of fertilizer each month per acre ... 10mg, 20 mg, 30 mg of a drug … 30db, 40 db, 70 db of music played during instruction

5. PLACEBO • Placebo: A treatment that does not really have any level of the factor we are treating the subjects with • Often referred to a “dummy pill” .. That creates the illusion that a treatment has occurred, when it has not • It is a strategy that allows us to compare the response a subject has merely from the fact that they are undergoing a treatment …to the response from undergoing the real treatment

6. LURKING VARIABLES • What are they, and what are we gonna do about them • Recall – they were those nasty hidden explanatory elements that might really be at the root cause of some response variable that we cannot account for, know about or control • So, a well designed must havce a CONTROL GROUP with randomized assignments to each group(s).

7. PLACEBO EFFECT Placebo Effect – any dummy treatment that triggers a response form the subject (inanimate objects do not think, and psychologically react to placebos). An expectation for a response on the part of the subject or the experimenter can often create the perception of a observed response. Example- Treating Ulcers – Gastric Freezing – later shown to just be the Placebo Effect

8. CONTROL GROUP • Control Group – The group of subjects or experimental units that do not receive any treatment for the purposes of comparison. (Example: you might conclude the fertilizer made an incredible difference … only to later compare the harvest to the control group and see it is the same. Lurking variable? Maybe incredibly fantastic weather all season long. • “CONTROL” is the FIRST BASIC PRINCIPLE of experimental design! – BIG IDEA #1 • Without Control – Experiments in medical treatments are ALWAYS BIASED in favor of results showing a positive effect.

9. RANDOMIZATIONBIG IDEA #2 • Assigning Subjects to treatment groups • Matching of subgroups is helpful – Stratifying can only go so far in controlling Lurking Variables • Simplest way to ensure that the Lurking Variables are not responsible for the responses .. RANDOMIZATION … assignment by CHANCE ALONE

10. CORN A vs. CORN B • What if we wanted to see which variety of corn would grow better. • We plant Corn A in plot X • We plant Corn B in plot Y • How can we tell if the corn grew better due to the brand of corn? … or the soil, sun, etc. impacting the two plots X and Y? • RHETORICAL … you can’t!

11. 30 RATS • Number all the rats 01 through 30. • Use a table of random digits or a computer to generate random values until 15 rats have been selected. • Assign them to group A • Assign the remaining to group B • … or alternate group A then group B then group A, etc.

12. RANDOMIZED COMPARATIVE EXPERIMENTS • Randomization of assignments assures that experimental units in each group are relatively similar in all respects before treatment • Comparative Design ensures that influences other than the treatment operate equally on both (all) groups. • Therefore differences in response must be due to the treatment OR the random effects of chance

13. EXPERIEMNTAL UNITS & n • BIG IDEA #3 – Use a big enough n – sample size – so the effects of the variation even out. • The effects of chance do average out over the long-run – and with larger samples • If you only use ONE or a FEW units, then the effects of chance are magnified, and the link to the treatment is ore uncertain

14. PRINCIPLES OF EXPERIMENTAL DESIGN - SUMMARY • 1. CONTROL • 2. RANDOMIZING • 3. REPLICATION

15. STATISITICAL SIGNIFICANCE • Statistical Significance – When an observed effect is so large that it would very RARELY occur by chance • Are we saying “IMPOSSIBLE” to occur … NO • But YOU the student of statistics must embrace new interpretations of “POSSIBLE” ... Or “WILL HAPPEN” • Flipping a coin right now in front of you 20 times inb a row HEADS … leads you to conclude that SOMETHING IS FAKE … rather than a RARE EVENT ACTUALLY JUST HAPPENED! • But to be clear – WE NEVER PROVE anything in Statistics

16. COMLETELY RANDOMIZED • Completely Randomized – All experimental units are assigned at random among all treatment groups Treatment 1 Meter Group 1 20 Houses Compare electricity use Random Assignment Group 2 20 Houses Treatment 2 Chart Group 3 20 Houses Treatment 3 Control

17. DOUBLE-BLIND • Double-blind - Neither the subjects nor the experimenter are aware of what treatment is being received by the subjects • Example: I administer 3 types of pills to patients in three groups Treatment A, B and the Control Group. I do so by having someone else code the pills, and I record which patients revived which coded pill.

18. LACK OF REALISM • Lack of Realism – occurs in an experiment where the conditions, environment or situation does not realistically replicate the conditions that we want to study • Example: Do YOU or I behave realistically when you know you are being OBSERVED or EXPERIMENTED ON? • Example: High Center Rear Brake Lights .. I’ll call it the novelty effect – a lurking variable

19. MATHCED PAIRS DESIGN • Matched Pairs Design – An experimental design in which two units are blocked together to receive the two different treatments. • A subject might receive both treatments, one after the other. Several subjects might be involved to allow for A follows B on half, and B follows A on the other half. • Before and After experiments are also an example. Rate sweetness … FREEZE .. Rate sweetness again

20. BLOCK DESIGN • Block – a group of experimental units or subjects that are known before the experiment to be similar in some way that is expected to affect the response to the treatments • Block Design – the random assignment of units to treatments is carried out separately within each block