1 / 31

Beginner’s guide to randomisation

Research supported by TLRI. Beginner’s guide to randomisation. Experiments & the Randomisation Test: The difference between two medians Watch out for: The ‘chance is acting alone’ explanation

ardith
Download Presentation

Beginner’s guide to randomisation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research supported by TLRI Beginner’s guide to randomisation

  2. Experiments & the Randomisation Test: • The difference between two medians • Watch out for: • The‘chance is acting alone’ explanation • How we assess the plausibility of the ‘chance alone’ explanation –(test for ‘chance alone’)

  3. What does ‘chance alone’ look like? iNZightVIT Randomisation

  4. Did she brush her teeth? Formulate statement to test. Data (information at hand). Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.? Review the statement in 1. in light of 3. together with the data in 2. • 1. She has brushed her teeth. • 2. The toothbrush is dry. • 3. The-toothbrush-is-dry would be highly unlikely if she had brushed her teeth. • Therefore, it’s a fairly safe bet she has not brushed her teeth. • I have evidence that she has not brushed her teeth.

  5. Did she brush her teeth (2)? Formulate statement to test. Data (information at hand). Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.? Review the statement in 1. in light of 3. together with the data in 2. • 1. She has brushed her teeth. • 2. The toothbrush is wet. • 3. The-toothbrush-is-wet would NOT be surprising if she had brushed her teeth. • Therefore, she could have brushed her teeth. • I have no evidence that she has NOT brushed her teeth. Or she could have just run the brush under the tap.

  6. The Walking Babies Experiment Does a special exercise programme lower walking age? Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) were randomly assigned to one of two treatment groups.

  7. The Walking Babies Experiment Does a special exercise programme lower walking age? Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) were randomly assigned to one of two treatment groups. First walked without support:

  8. The Walking Babies Experiment Does it appear that these data provide evidence that the treatment is effective? Exercise Yes. Control 9 10 11 12 13 14 Age (months)

  9. Looking at the world using data is like looking through a window with ripples in the glass “What I see … is not quite the way it really is”

  10. The Walking Babies Experiment Is it possible that these babies’ walking ages have nothing to do with whether they undertook the exercises or not, . . . Exercise Control 9 10 11 12 13 14 Age (months)

  11. The Walking Babies Experiment . . . i.e., it doesn’t matter which group they were randomly assigned to, they would still have the same walking age, . . . Exercise Control 9 10 11 12 13 14 Age (months)

  12. The Walking Babies Experiment . . . and so the observed difference pattern is purely and simply the result of the luck-of-the-draw as to which babies just happened by chanceto be assigned to which group and nothing else? Exercise Yes. Under this scenario we say: “Chance is acting alone”. Control 9 10 11 12 13 14 Age (months)

  13. The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect)

  14. The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: (the exercise has no effect) Chance is acting alone • Is the ‘chance alone’ explanation simply notplausible? • Would our observed difference be unlikelywhen chance is acting alone? • How do we determine whether an observed difference is unlikely when chance is acting alone? • Answer: See what’s likelyand what’s unlikelywhen chance is acting alone.

  15. 2.25

  16. Our observed difference = 2.25 months Is chance alone likely to generate differences as big as our difference? Hardly ever, they’re almost always smaller. Tail proportion: roughly __% Re-randomisation distribution of differences (under chance alone)

  17. The Walking BabiesExperiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) • Our tail proportion of roughly __% means: • Roughly __ times out-of-a-1000 times we get a median difference of 2.25 months or more, when chance is acting alone. • Under chance alone, it would be highly unlikely to get a difference equal to or bigger than our observed difference of 2.25 months.

  18. The Walking BabiesExperiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) therefore, it probably isn’t. • Our tail proportion of roughly __% means: • Roughly __ times out-of-a-1000 times we get a median difference of 2.25 months or more, when chance is acting alone. • Under chance alone, it would be highly unlikely to get a difference equal to or bigger than our observed difference of 2.25 months. It’s a fairly safe bet chance is not acting alone. An observed difference of 2.25 months or greater is highly unlikely when chance is acting alone . . .

  19. The Walking BabiesExperiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) • We can rule out ‘chance is acting alone’ as a plausibleexplanation for the difference between the two groups. • We have evidence against ‘chance is acting alone’ • We have evidence that chance is not acting alone

  20. The Walking BabiesExperiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) If chance is not acting alone, then what else is also acting to help produce the observed difference? Remember: Random assignment to 2 groups & each group receives different treatment. • We can rule out ‘chance is acting alone’ as a plausibleexplanation for the difference between the two groups. • We have evidence against ‘chance is acting alone’ • We have evidence that chance is not acting alone

  21. The Walking BabiesExperiment Conclusion: Because the male infants (& parents) were randomly assigned to the groups, we may claim that the exercise was effective in lowering the walking age. Because these subjects in this experiment were volunteers (not randomly selected), then we would need to consider carefully as to which wider group(s) this conclusion may apply.

  22. Did she brush her teeth? 1. She has brushed her teeth. 2. The toothbrush is dry.3. The-toothbrush-is-dry would be highly unlikely if she had brushed her teeth. 4. Therefore, it’s a fairly safe bet she has not brushed her teeth. I have evidence that she has not brushed her teeth. Steps Statement to test. Collect data (information). Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.? Review the statement in 1. in light of 3. together with the data in 2.

  23. Is the exercise programme effective? Steps Statement to test. Collect data. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2. or more? Review the statement in 1. in light of 3. together with the data in 2. • 1.Chance is acting alone.(The exercise has no effect.) • Diff between medians = 2.25 mths. • A difference of 2.25 months or more is highly unlikely when chance is acting alone.(Tail prop = roughly __%) • Therefore, it’s a fairly safe bet chance is not acting alone. We have evidence against ‘chance is acting alone’.

  24. Is the exercise programme effective? Steps Statement to test. Collect data. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2. or more? Review the statement in 1. in light of 3. together with the data in 2. 1. Chance is acting alone. (The exercise has no effect.) 2. diff medians= 2.25 mths. 3. A diff of 2.25 mthsor more is highly unlikely when chance is acting alone. 4. Therefore, it’s a fairly safe betchanceis not acting alone. We have evidence against chance is acting alone.

  25. Questions…

  26. Two types of Inference There are two types of inference • Sample-to-population egx = 172cm so the population mean is about 172cm. • Experiment-to-causation eg The treatment was effective

  27. Guidelines for assessing ‘Chance alone’ • When the tail proportion is small(less than 10%).: • the observed difference would be unlikely when chance is acting alone . . . therefore, it’s a fairly safe bet chance is not acting alone. • we have evidence against ‘chance-is-acting-alone’ • we have evidence that chance is not acting alone

  28. Guidelines for assessing ‘Chance alone’ • When the tail proportion is large(10% or more )then: • the observed difference is not unusual when chance is acting alone, therefore chance could be acting alone • we have NOevidence against ‘chance is acting alone’ • EITHER chance couldbe acting alone OR something else as well as ‘chance’ COULD also be acting. • -- we do NOT have ENOUGH INFORMATION to MAKE A CALLas to which one.

More Related