Chapter 4 Experiments and Observational Studies

1. Chapter 4Experiments and Observational Studies AP Statistics

2. Cause and Effect • Is it ever possible to provea cause-and-effect relationship? • Yes, it is, but we would have to take a different approach than what we’ve used before – regression, by itself, can’t prove causation. To prove cause-and-effect, we need to use an experiment. • An experiment is a controlled study in which the researcher attempts to understand cause-and-effect relationships. The study is "controlled" in the sense that the researcher controls (1) how subjects are assigned to groups and (2) which treatments each group receives.

3. Vocabulary • Experimental Units: The individuals on which the experiments are done. • Explanitory Variable:The variable that causes changes in another variable (the Response Variable) • Factor: An explanatory variable manipulated by the experimenter. • Levels: Different values of the factor. Each factor must have two or more levels. • Subjects: Human experimental units. • Treatment: A specific experimental condition applied to the units. The combinations of factor levels are called treatments.

4. Vocabulary Confounding Variable: Variable whose effect on the response variable is hopelessly mixed up with the explanatory variable. Lurking Variable: Variable with an explanatary/response association. A lurking variable effects both the explanatory and response variables. (can be thought of as an underlying cause)

5. Plyometrics: An Experiment • Suppose you are trying to advise the head coach of the boys’ basketball team on whether or not he should implement a plyometric training program. Plyometrics supposedly increase fast-twitch muscles making athletes quicker, stronger, faster, and more explosive. For simplicities sake, we will concentrate on quickness. • If there is a difference in performance, we would like to know if the difference is statistically significant; that is, if the observed effect is so large that it would rarely occur by chance. • What would be the experimental units and treatments of this experiment?

6. Principles of Experimental Design • The following three principles are components of all valid experiments: • Control: We control sources of variation other than the factors we are testing making conditions as similar as possible for all treatment groups. Basically, we are trying to eliminate as many lurking variables as possible. • Randomize: The practice of using chance methods to assign subjects to treatments. Randomization allows us to equalize the effects of unknown or uncontrollable sources of variation. • Replicate: The practice of assigning each treatment to many experimental subjects. Replication allows us to make comparisons between groups and to capture data that is representative of the population.

7. Plyometrics: An Experiment • We want to determine whether a plyometrics program should be established at VCS. You decide to set up an experiment to see whether the plyometrics increase athletic performance. • You and others probably have an opinion already, so your opinion might incorrectly influence the record-keeping and the outcomes. To the best of our ability, we need to try and eliminate such biases. • To avoid bias, you would need to disguise the factors as much as possible. This strategy is called blinding– a technique to insure those who participate in an experiment do not know who is getting which treatment. How can you blind our plyometric experiment?

8. Single Blinds and Double Blinds • There are two classes of individuals who can affect the outcome of the experiment • Those who could influence the results (the subjects, administrators, or the technicians) • Those who evaluate the results (judges, treating physicians, etc.) • When all the individuals in either one of these classes is blinded, an experiment is said to be a single-blind. When everyone in both classes is blinded, we call the experiment a double-blind.

9. The Placebo Effect Placebo:A dummy treatment given in an experiment. ex: Giving one group sugar pills when testing the effect of pain killers on headaches. Placebo Effect:The positive response to the dummy treatment. ex: The patients in the placebo group don’t have headaches after taking the sugar pills.

10. Statistical Studies • There are basically two types of studies that are conducted within statistics: observational studies and experiments. • An observational studyobserves individuals and measures a variable of interest but does not attempt to influence the responses. • A special type of observational study is a survey. A sample surveyis a study that asks questions of a sample drawn from some population in the hope of learning something about the entire population. • An experiment, on the other hand, deliberately imposes some treatment on individuals in order to observe their responses.

11. You Can Observe A Lot Just By Watching… • Observational studies attempt to determine associations between variables. We would like to show a cause-effect relationships, but that is not always possible. In order to show a cause-effect relationship, the researchers need use experiments. • However, there are several advantages of an observational studies over experiments. Observational studies • Preserve the integrity of experimental units, keeping them unharmed, intact, and unaffected • Are often less costly and time-consuming • Avoid ethical dilemmas and artificiality of experiments

12. You Can Observe A Lot Just By Watching… • The main disadvantage of observational studies is the potential threat of lurking variables. Often times, we may see a strong association between our variables but some lurking variable may be confounding our cause-and-effect relationship • For example, we may notice that there is a very strong association between drinking alcohol and lung cancer. However, this association is confounded by smoking (since, within the study, many of those who engaged in drinking also engaged in smoking). We are unsure if the drinking, smoking, or combination of the two actually causes cancer.

13. Experiment Design: “The 4th Principle” • The 4th principle of experimental design is blocking,which allows us to make a compromise between randomization and control. Although it is not a required principle, it can remove much of the variability caused by individual differences among our subgroups. • Block: (def) a group of experimental units or subjects that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. • Blocking Variable: A variable which allows individuals in an experiment to be arranged into blocks.

14. Experiment Design: “The 4th Principle” • Randomized Blocking Design: An experiment where similar individuals are placed in blocks, then subjects within each block are randomly assigned to treatment conditions. By blocking we isolate the variability attributable to the differences between the blocks, so that we can see the differences caused by the treatments more clearly.

15. Plyometrics: An Experiment • Going back to the plyometric experiment, should we use a randomized blocking design? Do you think there might be a difference between people who are very quick and those who are not? Is there a way to break them up into groups? Is it necessary? • Are there other reasons that we might break the subjects into different groups such as ethnicity, class, athleticism, team (Varsity, JV, Frosh), etc.? Is it easy to distinguish who belongs in each group?

16. Completely Randomized Design • If there is no reason or it is impractical to block your experiment, you may wish to use a simpler design, such as the completely randomized design (this is the simplest kind of experiment). • Completely Randomized Design: A design where all experimental units have an equal chance of receiving any treatment.

17. Designing an Experiment • Let’s create a design for our plyometric experiment. Let’s use the completely randomized experiment design. • Plan:State what you want to know • I want to know whether plyometrics will increase athletic performance – specifically, I wish to determine if plyometrics will increase quickness in otherwise similar circumstances but without the plyometrics. • Response:Specify the response variable • I will evaluate the quickness of each athlete utilizing a timer and several quickness courses and drills. • Treatments:Specify the factor levels and treatments • The factor is plyometrics. I will evaluate the plyometrics at three levels: some athletes will not perform any plyometrics, some athletes will perform half the repetitions of the plyometric program, and some will complete the full plyometric system.

18. Designing an Experiment • Experimental Units:Specify the experimental units • I will use 24 basketball players from the Village Christian boys’ basketball program. Now apply the principles of experimental design! • Controlany sources of variability you know and can control • I will create a detailed plyometric program that athletes within each group will follow exactly. The exercises in the program will be exact and explicit. Athletes will perform all exercises in a central location and plyo-boxes will be used to ensure a consistent jumping height.

19. Designing an Experiment • Randomly assignexperimental units to treatments, to equalize the effects of unknown or uncontrollable sources of variation • I’ll randomly assign each athlete to one of the three groups. I will use random numbers from the table to determine the assignment of each athlete. • Replicateresults by placing more than one athlete in each treatment group • There will be eight athletes in each treatment group. • Make a diagram

20. Designing an Experiment • Specify any other experimental details. You must give enough details so that another experimenter could exactly replicate your experiment. It is generally better to include details that might seem irrelevant than to leave out matters that could turn out to make a difference. • The participating athletes will perform plyometrics three days a week for a period of 12 weeks. The program should run during the “off-season” to limit and control the amount of activity each athlete performs. • Specify how to measure the response. • Quickness will be measured by timing how long it takes each athlete to complete three quickness drill. Quickness drills will include a ladder course, “lines,” and the 40-yard dash.

21. Designing an Experiment • Once you collect the data, you’ll need to display them and compare the results for the three treatment groups. • I will display the results with side-by-side boxplots to compare the three treatments groups. I will compare the means of the groups and record any differences. • To answer the initial question, we ask whether the differences we observe in the means of the three groups are meaningful. Because this is a randomized experiment, we can attribute significant differences to the treatments. To do this properly, we’ll need methods from what is called “statistical inference,” a topic that we will cover in-depth later on. • If the difference in quickness is greater than I would expect by usual variance, I may be able to conclude that these differences can be attributed to treatments with a plyometric program.

22. Can We Do Better? • Although one group may do significantly better than the others, does that mean that it is because of the plyometrics? It’s possible, but it is also possible that one group just has quicker athletes. How can we limit this kind of variance? • We should use a block design. • What attribute should we use to block the athletes such that we eliminate much of the variance? • The blocks could be based on quickness, speed, agility, strength, etc. • How will we conduct our random assignment? • To assign each athlete, we will first separate our athletes into blocks, then we would randomly assign athletes from each block into our three levels of plyometrics.

23. Matched-Pairs: A Special Block Design • Matched pair, a special kind of block design, can help us remove much of the variance when the experiment has only twotreatment conditions. • In a matched pair design, subjects are grouped into pairs, based on some blocking variable. Then, within each pair, subjects are randomly assigned to different treatments. • Like the completely randomized design, the matched pairs design uses randomization to control for confounding. However, unlike the other design, the matched pairs design explicitly controls for potential lurking variables. • If the order is varied, the experiment unit can serve as its own control.

24. Last Thoughts • Sometimes you want to make a comparison when a treatment is applied and not applied. Such baseline measurements are called control treatments, and the experimental units to whom it is applied are called control groups. • This is a use of the word “control” in an entirely different context. Previously, we controlled extraneous sources of variation by keeping them constant. Here, we use a control treatment as another level of the factor in order to compare the treatment results to a situation in which “nothing happens.” In our plyometric example, the group that did not perform the plyometrics would be considered our control group.

25. Last Thoughts • If experimental units are not assigned to treatments at random, you will not be able to use the powerful methods of Statistics to draw conclusions from your study – randomization is imperative. • It is fine if the blocks in our study are not exactly even. In other words, each block can have a different number of participants • The following slide has an example of a more complicated design using blocking:

26. Diagramming a Blocked Experiment