Experiments vs. Observational Studies

1. Experiments vs. Observational Studies • Experiments • Observe responses to variables • Administer a treatment in order to observe the response to the treatment • Can determine causation • Observational Studies • Observe responses to variables • Simply observes responses, no attempt to influence them • Can NOT determine causation (only correlation)

2. Causation • This may be the most significant reason for conducting an experiment as opposed to an observational study. • Most often, people are interested in causation. • For example: • Will this drug cause my headache to go away? • Does sending my child to daycare cause my child to be behind later in life? • Does making students pass a test to graduate cause improvement in our nation’s education? • Will invading Iraq cause long-term peace in that country? • Unfortunately, experiments (which must control for other variables) are simply not possible to conduct. • When it is possible, it is the way to go!

3. Experiments and Terminology • Experimental Units • The things that the experiment is done on • Also called subjects (when they are human) • Treatment • What is actually done to the experimental units • Factors • The different types of treatments (which are the explanatory variables) • Levels • Differing amounts of the treatment or factors

4. Experiments and terminology • How do the terms apply? • Consider the experiment: • A consumer advocacy group is curious about the effectiveness of pain medication in treating migraine headaches. They randomly give different doses of aspirin, tylenol, and ibuprofen to migraine sufferers. They then measure the results and compare. • Apply the terms experimental unit, treatment, factor and level to the scenario above.

5. Experiments and terminology • What were the experimental units? • The migraine patients (since human, subjects) • What was the treatment? • The pain medication • What were the factors? • The aspirin, tylenol, and ibuprofen • What were the levels? • The dosages of the drugs

6. EXAMPLE The Characteristics of an Experiment The English Department of a community college is considering adopting an online version of the freshman English course. To compare the new online course to the traditional course, an English Department faculty member randomly splits a section of her course. Half of the students receive the traditional course and the other half is given an online version. At the end of the semester, both groups will be given a test to determine which performed better. (a) Who are the experimental units? The students in the class All students who enroll in the class (b) What is the population for which this study applies? (c) What are the treatments? Traditional vs. online instruction (d) What is the response variable? Exam score (e) Why can’t this experiment be conducted with blinding? Both the students and instructor know which treatment they are receiving

7. More terminology • Just like bivariate data, there is typically an explanatory variable and a response variable in an experiment. • Consider the scenario: • A teacher wants to see if a new computer program can more effectively increase the reading ability of students than a traditional classroom setting. She first tests each student from a 4th grade class. She then randomly selects half of the class to participate in the computer program and the other half in the traditional curriculum. At the end of the year, she tests the students reading ability again and compares the results. • Identify the explanatory variable. • Whether they received the computer program or traditional curriculum • Identify the response variable. • The difference in the results of the reading ability tests.

8. Comparative Experiments • Most experiments are comparative. • That is, the purpose of the experiment is to compare a treatment to a lack of treatment or to compare two or more treatments.

9. Administer Treatment Observe Results Administer Treatment #1 Administer Treatment #2 Observe Results & Differences Administer Treatment #3 Observe Response Variable Administer Treatment Observe Response Variable Experimental Design Overview • Simple Experiments (3 models)

10. Nature of Experiments • The nature of an experiment is to focus in on causation. • This is done by controlling variables. • Variables are controlled through randomization and the use of control groups.

11. A completely randomized design is one in which each experimental unit is randomly assigned to a treatment.

12. Completely Randomized Experiment • A graphical model is typically used for designing experiments. • Consider the question, “Does smoking cause lung cancer?” • Unfortunately, a direct experiment would be unethical, but here is what it would look like.

13. Smoking and Lung Cancer • Start with, say, 400 volunteers who have never smoked before. • Randomly choose 200 of them for the experimental group and the other 200 form the control group. • The treatment is to smoke 1 pack of cigarettes per day. • The control group does not smoke at all. • Then, track the volunteers for, say 40 years. • At the end of the 40 years, count up how many in each group developed lung cancer. • This is a good description of the experiment, the next slide shows the same thing in diagram form.

14. Smoking and Lung Cancer Must state HOW to randomly allocate – use a hat Experimental Group (200) Smoke 1 pack of cigarettes per day for 40 years Measure Data Did lung cancer develop? 400 Volunteers Observe Results Draw Conclusions Randomly Allocate Control Group (200) No smoking for 40 years Measure Data Did lung cancer develop?

15. EXAMPLEDesigning an Experiment The octane of fuel is a measure of its resistance to detonation with a higher number indicating higher resistance. An engineer wants to know whether the level of octane in gasoline affects the gas mileage of an automobile. Assist the engineer in designing an experiment.

16. Completely Randomized Design