13 Collecting Statistical Data

2. 13 Collecting Statistical Data 13.1 The Population 13.2 Sampling 13.3 Random Sampling 13.4 Sampling: Terminology and Key Concepts 13.5 The Capture-Recapture Method 13.6 Clinical Studies

3. Survey A survey typically deals with issues and questions that have direct and measurableanswers. If the election were held today, would you vote for candidate X or candidate Y? How many people live in your household? How many catfish have tags?In these situations data collection involves some combination of observing,measuring, and recording but no active involvement or interference with thephenomenon being observed.

4. Cause and Effect A different type of data collection process is needed when we are trying to establish connections between a cause and an effect. Does taking a math class increaseyour chances of getting a good paying job? Does repeated exposure to secondhandsmoke significantly increase your risk for developing lung cancer? Does a daily doseof aspirin reduce your chances of a heart attack? Do the benefits of hormone replacement therapy for women over 50 outweigh the risks?

5. Cause and Effect These kinds of cause-and-effectquestions cannot be answered by means of an immediate measurement and requireobservation over an extended period of time. Moreover, in these situations the datacollection process requires the active involvement of the experimenter–in additionto observation, measurement, and recording, there is also treatment.

6. Cause and Effect When we want to know if a certain cause X produces a certain effect Y, we setup a study in which cause X is produced and its effects are observed. If the effect Yis observed, then it is possible that X was indeed the cause of Y. We have established an association between the cause X and the effect Y. The problem, however,is the nagging possibility that some other cause Z different from X produced the effect Y and that X had nothing to do with it.

7. Cause and Effect Just because we established an association, we have not established a cause-and-effect relation between the variables.Statisticians like to explain this by a simple saying: Association is not causation.

8. CASE STUDY 5 THE ALAR SCARE Alar is a chemical used by apple growers to regulate the rate at which applesripen. Until 1989, practically all apples sold in grocery stores were sprayed withAlar. But in 1989 Alar became bad news, denounced in newspapers and on TV asa potent cancer-causing agent and a primary cause of cancer in children. As a result of these reports, people stopped buying apples, schools all over the countryremoved apple juice from their lunch menus, and the Washington State apple industry lost an estimated $375 million.

9. CASE STUDY 5 THE ALAR SCARE The case against Alar was based on a single 1973 study in which laboratorymice were exposed to the active chemicals in Alar. The dosage used in the studywas eight times greater than the maximum tolerated dosage–a concentration atwhich even harmless substances can produce tissue damage. In fact, a childwould have to eat about 200,000 apples a day to be exposed to an equivalentdosage of the chemical.

10. CASE STUDY 5 THE ALAR SCARE Subsequent studies conducted by the National CancerInstitute and the Environmental Protection Agency failed to show any cause-and-effect relationship between Alar and cancer in children.While it is generally accepted now that Alar does not cause cancer, becauseof potential legal liability, it is no longer used.

11. CASE STUDY 5 THE ALAR SCARE The Alar scare turned out to be afalse alarm based on a poor understanding of the statistical evidence. Unfortunately, it left in its wake a long list of casualties, among them the apple industry,the product’s manufacturer, the media, and the public’s confidence in the system.For most cause-and-effect situations, especially those complicated by theinvolvement of human beings, a single effect can have many possible and actualcauses.

12. CASE STUDY 5 THE ALAR SCARE What causes cancer? Unfortunately, there is no single cause–diet, lifestyle,the environment, stress, and heredity are all known to be contributory causes. Theextent to which each of these causes contributes individually and the extent towhich they interact with each other are extremely difficult questions that can beanswered only by means of carefully designed statistical studies.

13. Clinical Study We will illustrate an important type of studycalled a clinical study or clinical trial. Generally, clinical studies are concerned withdetermining whether a single variable or treatment (usually a vaccine, a drug,therapy,etc.) can cause a certain effect (a disease, a symptom, a cure, etc.). The importance of such clinical studies is self-evident: Every new vaccine, drug, or treatment must prove itself by means of a clinical study before it is officially approvedfor public use.

14. Clinical Study Likewise, almost everything that is bad for us (cigarettes, caffeine,trans fats, etc.) gets its official certification of badness by means of a clinical study. Properly designing a clinical study can be both difficult and controversial, andas a result we are often bombarded with conflicting information produced by different studies examining the same cause-and-effect question. The basic principlesguiding a clinical study, however, are pretty much established by statistical practice.

15. Isolate the Cause The first and most important issue in any clinical study is to isolate the cause(treatment, drug, vaccine, therapy, etc.) that is under investigation from all otherpossible contributing causes (called confounding variables) that could producethe same effect. Generally, this is best done by controlling the study.

16. Controlled Study In a controlled study, the subjects are divided into two different groups: thetreatment group and the control group.The treatment group consists of those subjects receiving the actual treatment; the control group consists of subjects that arenot receiving any treatment–they are there for comparison purposes only (that’swhy the control group is sometimes also called the comparison group).

17. Controlled Study If a realcause-and-effect relationship exists between the treatment and the effect beingstudied, then the treatment group should show the effects of the treatment andthe control group should not. To eliminate the many potential confounding variables that can bias its results, a well-designed controlled study should have control and treatment groupsthat are similar in every characteristic other than the fact that one group is beingtreated and the other one is not.

18. Randomized Controlled Study Themost reliable way to get equally representative treatment and control groups is to use a randomized controlled study. In a randomized controlled study, the subjectsare assigned to the treatment group or the control group randomly. When the randomization part of a randomized controlled study is properlydone, treatment and control groups can be assumed to be statistically similar.

19. Placebo Effect Butthere is still one major difference between the two groups that can significantly affect the validity of the study–a critical confounding variable known as the placeboeffect.The placebo effect follows from the generally accepted principle that just theidea that one is getting a treatment can produce positive results.

20. Placebo Thus, when subjects ina study are getting a pill or a vaccine or some other kind of treatment, how can theresearchers separate positive results that are consequences of the treatment itselffrom those that might be caused by the placebo effect? When possible, the standardway to handle this problem is to give the control group a placebo.

21. Controlled Placebo Study A placebo is amake-believe form of treatment–a harmless pill, an injection of saline solution, orany other fake type of treatment intended to look like the real treatment. A controlled study in which the subjects in the control group are given a placebo is calleda controlled placebo study.

22. Controlled Placebo Study By giving all subjects a seemingly equal treatment (the treatment group gets thereal treatment and the control group gets a placebo that looks like the real treatment), we do not eliminate the placebo effect but rather control it–whatever its effect might be, it affects all subjects equally. It goes without saying that the use ofplacebos is pointless if the subject knows he or she is getting a placebo.

23. Blind Study Thus, a second key element of a good controlled placebo study is that all subjects be kept in thedark as to whether they are being treated with a real treatment or a placebo. A studyin which neither the members of the treatment group nor the members of the control group know to which of the two groups they belong is called a blind study.

24. Double-Blind Study To keep the interpretation of the results (which can often be ambiguous) totally objective, it is important that the scientists conducting the study and collectingthe data also be in the dark when it comes to who got the treatment and who gotthe placebo. A controlled placebo study in which neither the subjects nor the scientists conducting the experiment know which subjects are in the treatmentgroup and which are in the control group is called a double-blind study.

25. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS Polio (infantile paralysis) has been practically eradicated in the Western world. Inthe first half of the twentieth century, however, it was a major public health problem. Over one-half million cases of polio were reported between 1930 and 1950,and the actual number may have been considerably higher.Because polio attacks mostly children and because its effects can be so serious (paralysis or death), eradication of the disease became a top public healthpriority in the United States.

26. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS By the late 1940s, it was known that polio is a virusand, as such, can best be treated by a vaccine that is itself made up of a virus. Thevaccine virus can be a closely related virus that does not have the same harmfuleffects, or it can be the actual virus that produces the disease but that has beenkilled by a special treatment. The former is known as a live-virus vaccine, the latter as a killed-virus vaccine.

27. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS In response to either vaccine, the body is known toproduce antibodies that remain in the system and give the individual immunityagainst an attack by the real virus.Both the live-virus and the killed-virus approaches have their advantages anddisadvantages. The live-virus approach produces a stronger reaction and betterimmunity, but at the same time, it is also more likely to cause a harmful reaction and, in some cases, even produce the very disease it is supposed toprevent.

28. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS The killed-virus approach is safer in terms of the likelihood ofproducing a harmful reaction, but it is also less effective in providingthe desired level of immunity. These facts are important because they help us understand the extraordinary amount of caution that went into the design of the study thattested the effectiveness of the polio vaccine.

29. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS By 1953, several potentialvaccines had been developed, one of the more promising of which was akilled-virus vaccine developed by Jonas Salk at the University of Pittsburgh. The killed-virus approach was chosen because there was a greatpotential risk in testing a live-virus vaccine in a large-scale study.

30. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS The testing of any new vaccine or drug creates many ethical dilemmasthat have to be taken into account in the design of the study. With a killed-virus vaccine the risk of harmful consequences produced by the vaccine it-self is small, so one possible approach would have been to distribute thevaccine widely among the population and then follow up on whetherthere was a decline in the national incidence of polio in subsequent years.

31. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS This approach, which was not possible at the time because supplies were limited, iscalled the vital statistics approach and is the simplest way to test a vaccine. This is essentially the way the smallpox vaccine was determined to be effective. The problemwith such an approach for polio is that polio is an epidemic type of disease, whichmeans that there is a great variation in the incidence of the disease from one year tothe next.

32. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS In 1952, there were close to 60,000 reported cases of polio in the UnitedStates, but in 1953, the number of reported cases had dropped to almost half that(about 35,000). Since no vaccine or treatment was used, the cause of the drop wasthe natural variability typical of epidemic diseases.

33. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS But if an ineffective polio vaccinehad been tested in 1952 without a control group, the observed effect of a large dropin the incidence of polio in 1953 could have been incorrectly interpreted as statistical evidence that the vaccine worked.The final decision on how best to test the effectiveness of the Salk vaccinewas left to an advisory committee of doctors,public officials,and statisticians convened by the National Foundation for Infantile Paralysis and the Public HealthService.

34. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS Approximately750,000 children were randomly selected to participate in the study. Of these,about 340,000 declined to participate, and another 8500 dropped out in themiddle of the experiment. The remaining children were randomly divided intotwo groups–a treatment group and a control group–with approximately 200,000children in each group.

35. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS Neither the families of the children nor the researcherscollecting the data knew if a particular child was getting the actual vaccine or ashot of harmless solution. The latter was critical because polio is not an easy diseaseto diagnose–it comes in many different forms and degrees. Sometimes it can be aborderline call, and if the doctor collecting the data had prior knowledge ofwhether the subject had received the real vaccine or the placebo, the diagnosiscould have been subjectively tipped one way or the other.

36. CASE STUDY 6 THE 1954 SALK POLIO VACCINE FIELD TRIALS A summary of the results of the Salk vaccine field trials is shown on the next slide.These data were taken as conclusive evidence that the Salk vaccine was an effectivetreatment for polio, and on the basis of this study, a massive inoculation campaignwas put into effect. Today, all children are routinely inoculated against polio, andthe disease has essentially been eradicated in the United States. Statistics played akey role in this important public health breakthrough.