Statistics 100 Lecture Set 4

1. Statistics 100Lecture Set 4

2. Lecture Set 4 • Chapter 5 and 6 … please read • Read Chapter 7 … you are responsible for all of this chapter • Some suggested problems: • Chapter 4: 4.9, 4.17, 4.21, 4.29 • Chapter 5: 5.3, 5.7, 5.11, 5.15, 5.17, 5.25 • Chapter 6: 6.5, 6.9, 6.11, 6.15, 6.21 • Chapter 7: 7.7, 7.9, 7.13, 7.21

3. Observational studies • Generally, we have been discussing observational studies • These methods look at the current population and try and extract information • No attempt is made to change anything or influence the response • Note: a sample survey is a type of observational study

4. Observational studies • Observational studies often subject to the presence of lurking variables

5. Example • A quick glance at the weekly ice cream sales in the summer months shows that when sales are high, deaths in recreational boating accidents is also high. Similarly, when sales are low, deaths in recreational boating accidents is also low • What does this tell us?

6. Experiments • Designed experiments: we modify the factors of interest and see the response • By manipulating the explanatory variables, we are trying to establish a cause-and-effect relationship between a set of variables and the response of interest

7. Some definitions • Response variable: • Explanatory variable: • Treatment:

8. Example • A study deliberately exposes rats to cigarette smoke to see whether they are more prone to develop lung tumours than unexposed rats • Response variable: • Explanatory variable: • Treatment:

9. Experiments • Lurking variables are not only found in observational studies • This can occur in experiments as well, if you are not careful • A properly designed experiment should attempt to account for potential lurking variables

10. Experiments • How should treatments be assigned to subjects? • Self-Selection: subjects choose own treatment • Systematic: assign to subjects based on some characteristic • Random: Use a randomizing device to assign them

11. Experiments • What is the goal the treatment assignment? • What do we want our groups to be like when we are done with assigning the treatments? • So what do we want to achieve by the assignment?

12. Experiments • Goal is to learn how treatment impacts the response • Want treatment to be only real difference between groups • Need groups to be as similar as possible • No lurking variables favouring one group over another • Note: It is called confounding when effects of two variables cannot be distinguished from each other

13. Back to treatment assignment • Self Selection • Example: exercise vs. medicine for arthritis • Patient decides upon the treatment

14. Back to treatment assignment • Systematic Assignment • Assign to subjects based on some characteristic (say men in one group women in the other)

15. Back to treatment assignment • Random Assignment • Use external device to determine who gets each treatment

16. Experiments • So, we have to be careful about how we assign subjects to treatments • What about the treatments • Often there is one main thing of interest • A new drug • Smoking • Online learning • Can we run an experiment with just one treatment?

17. Experiments • One treatment experiments, in addition to being less informative, often suffer from a certain type of confounding called

18. Randomized Comparative Experiments • The aim of an experiment is to uncover the effect of an explanatory variable on the response variable

19. Randomized Comparative Experiments • Need to have at least two treatments in a comparative experiment • A Control is a treatment representing something standard. • Give a patient a placebo (if doing nothing is OK) • Use current best practice (cancer studies) • Gives you something to compare the “new” treatment against

20. Randomized Comparative Experiments • Researcher Bias • If you invented a new thing, don’t you hope it works? • If someone pays you to try something for them, don’t you want their thing to work? • Blinding prevents these biases • Double blind studies are those where both the researcher and the subject do not know which treatment is assigned to each subject… done in an to eliminate subjective bias from both subjects and the researchers

21. Randomized Comparative Experiments Experiments are designed as carefully as surveys • Use individuals who represent the population • Randomize them to treatments • Control against confounding by lurking variables • Analyze treatment comparisons using good statistics • Use enough subjects to reduce the impact of chance variation

22. Experiments • Basic principles:

23. Experiments • How to interpret results? • Why was the study performed?

24. Statistical significance • Statistical significance:

25. Statistical significance • Suppose that you read a newspaper report that said: • a study at SFU found that there was a statistically significant difference between the obesity rates of breast fed infants compared to those who are not breast fed • What does this mean?

26. Statistical significance • We use statistical methods to determine when an observed effect is statistically significant • Major area for statistical practice and research

27. Start with the assumption that all treatments impact the response in the same way (there is no difference) Collect data and compute a statistic that can compare the difference in treatments Assess the size of the effect Reject or do not that the hypothesis that there is no difference in the treatment responses Not guilty Guilty Collect evidence and present to jury Weigh evidence Decide if guilty or not guilty Similar to a jury trial

28. Statistical significance • If you reject the hypothesis that the effects are all the same, the treatments are said to be statistically significant • Do you ever PROVE that they are the same or different? • Notice you reject or do not reject …

29. What to do about observational studies? • Does smoking cause cancer? • The USA Surgeon General (The 1964 Report on Smoking and Health) warned about the link between lung cancer and cigarette smoking • Why did it take years until tobacco companies finally stopped battling this claim?

30. What to do about observational studies? • Establishing causation from observational studies

31. Chapter 6: Experiments in the real world • Suppose that we could do an experiment studying smoking in humans. • How would we conduct the study?

32. Chapter 6: Experiments in the real world • Experiments require a lot of careful planning and knowledge • Goal is to select subjects and assign treatments to minimize bias and variability of comparisons… and to estimate the effects of interest as efficiently as possible

33. Chapter 6: Experiments in the real world • How could we form similar groups of individuals?

34. Chapter 6: Experiments in the real world • Problem: Variability in experimental materials • Subjects are variable • Not all have the same potential for response • Conditions are variable • Educational outcomes from schools • Different regions in a clinical trial • Treatments have characteristics other than what is being studied • Macs versus PCs

35. Chapter 6: Experiments in the real world • Try to avoid the variability in subjects

36. Chapter 6: Experiments in the real world • Try to avoid the variability in conditions

37. Experiment 1: 10 boys were selected at random Each boy was given a pair of shoes 5 boys received a pair of shoes with the old sole (Sole A) and 5 boys received shoes with the the new sole (sole B) Each boy wears the shoes for 1 month and the amount of wear is measured Experiment 2 10 boys were selected at random Each boy was given a pair of shoes Each pair had 1 shoe with the old sole and 1 shoe with the new sole For each pair of shoes, the sole type was randomly assigned to the right or left foot Each boy wears the shoes for 1 month and the amount of wear is measured Quick Example: Are These the Same?

38. Data

39. Plot of Responses by Treatment

40. Plot of Responses by Treatment

41. Some types of experiments • Completely randomized experiments:

42. Example • Suppose have 4 different diets which we want to compare (A,B,C and D) • Interested in how the diets affect the weight gain in rats undergoing chemotherapy • Have 16 rats available for the experiment – 4 for each diet • How should we use randomization to assign rats to treatments • Note: The rats arrive and are placed in a pen until the experiment begins, at which time they will be placed into cages

43. Example • How about this?

44. Example • How about this?

45. Example • How about this?

46. Randomized block designs • A block is a group of experimental subjects that are know beforehand to be similar in some way expected to impact the response • In a randomized block design, the treatments are randomly assigned to units with the block

47. Example • Suppose have 4 different diets which we want to compare (A,B,C and D) • Interested in how the diets affect the weight gain in rats undergoing chemotherapy • Have 16 rats available for the experiment – 4 for each diet • How should we use randomization to assign rats to treatments • Suppose cages are arranged on 4 shelves…4 cages/shelf • Note: The rats arrive and are placed in a pen until the experiment begins, at which time they will be placed into cages

48. Example • How about this?

49. Experiments in the real world • Problem: Placebo Effect • Pretending to cure often helps (jury is out on this) • Giving fake pills • Using machine that doesn’t do anything • If a real treatment shows positive effect, how do you know that the treatment is helpful beyond this?

50. Experiments in the real world • Way to deal with this is not to tell the patient that they are getting the placebo • Called blinding • Can interpret the experiment in terms of the comparison with the placebo • When a placebo is not possible, use …