EXPERIMENTS

1. EXPERIMENTS Introduction

2. Observational Study vs. Statistical Experiment Observational Study: a study in which two ‘groups’ are simply observed. Eg. Observational study on the effects of caffeine: People will be asked whether or not they consumed caffeine, and may have done a reaction time test online. Compare the two groups. See if one group tends to have a smaller reaction times than the other? Experimental Study: is one in which two groups are compared, but the statistician tries to control as much as possible, including who consumes caffeine, how much, etc. Eg. Experiment studying the effects of caffeine: Get half the class to consume caffeine soft-drinks The other half consumes non-caffeine soft drinks Measure Reaction Time Compare the two groups. Does caffeine cause a change in reaction time?

3. Observational Study vs. Statistical Experiment Observational Study: a study in which two ‘groups’ are simply observed. • What Problems Could Arise from this Type of Study compared to an Experiment? • Whether a person has drunk heaps or a little caffeine, they would simply be grouped under the “have had caffeine” group • People may not realise things like coke have caffeine. • If you don’t administer the reaction test yourself, not everyone would have the same power computer/ some may have gaming mouses. • Some may not take the test seriously, but in an experiment this is less likely. • etc.

4. Observational Study vs. Statistical Experiment Observational Study:Your Conclusion may look like this: I can make the call that the people who had caffeine had a lower reaction time than the people who didn’t have caffeine Notice we are saying there is a difference, but avoiding any allusion to what caused it Experimental Study: Your Conclusion may look like this: I can make the call that drinking caffeine causes a lower reaction time This is a much more powerful finding, as you are making a call not only that there is a difference, but what seems to have caused it

5. James Lind James Lind was a Scottish doctor back in the 1700s. He was a pioneer of naval hygiene in the British Royal Navy. He is famous for conducting one of the first ever scientifically planned experiments From this experiment he was able to establish the theory that eating citrus fruits cured scurvy. This not only saved thousands of lives, but also began efforts to formalise techniques for a robust clinical trial By Sir George Chalmers, c 1720-1791 - [1], Public Domain, https://commons.wikimedia.org/w/index.php?curid=32922810

6. Many sailors in the Royal Navy were dying from Scurvy, and James Lind was tasked with finding a solution There were many theories on how to treat scurvy, but how could these be separated into legitimate cures and simple old wives tales? He divided twelve scorbutic sailors into six groups of two. Lind limited his subjects to men who "were as similar as I could have them."

7. They all received the same diet but, in addition: • group one was given a quart of cider daily. • group two twenty-five drops of elixir of vitriol (sulfuricacid) • group three six spoonfuls of vinegar • group four half a pint of seawater • group five received two oranges and one lemon • the last group a spicy paste plus a drink of barley water. The treatment of group five stopped after six days when they ran out of fruit, but by that time one sailor was fit for duty while the other had almost recovered! The other groups noticed some relief from the scurvy, but nothing anywhere near as dramatic as the group that had citrus fruits. https://en.wikipedia.org/wiki/James_Lind

8. BASIC DESIGN IDEAS IN EXPERIMENTS

9. PAIRED COMPARISON VS TWO INDEPENDENT GROUPS Paired comparison: Twomeasurementsfrom the same person or object Two independent groups: Onemeasurementfrom each person/object in each group

10. Paired comparison Example Immediately, right now…write down how happy you are on a scale of 1 to 10 • 1  I am totallysad • 10  I am totallyhappy Your teacher will now ask for the number you wrote down and record it. Notice they do not need your name but only your response

11. Now watch the following video: https://youtu.be/Qj9KhfRmrow

12. Paired Comparison Example Immediately, right now, NOW! NOW! NOW!…write down how happy you are on a scale of 1 to 10 • 1  I am totally sad • 10  I am totally happy Your teacher will now ask for the number you wrote down and record it. Notice they do not need your name but only your response

13. Next Activity • Toss a coin • If you have a head, leave the room for five minutes

14. the rest of you put your pencil in your mouth and rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

15. the rest of you put your pencil in your mouth and rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

16. the rest of you put your pencil in your mouth and rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

17. the rest of you put your pencil in your mouth and rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

18. the rest of you put your pencil in your mouth and rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

19. Add up the total of the 5 scores and pass the total on to your teacher • Now let the people who had a tails come back in the room

20. Rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

21. Rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

22. Rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

23. Rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

24. Rate the following memes from 1 to 10 on funniness. 1 being totally not funny 10 being totally funny

25. This is an experiment using two independent groups

26. Paired comparison Twomeasurementsfrom the same person or object (MOTIVATIONAL VIDEO EXAMPLE) Two independent groups Onemeasurement from each person/object in each group (DISNEY MEMES EXAMPLE)

27. Choosing ‘UNITS’ • Units are the entities that will part-take in your experiment. • The units are often people or animals, but not always. • For higher marks one must be able to justify their choice of units. • Can anyone think of some reasons that justify the use of Year 12 students as units for a study on whether caffeine effects reaction time? • What about the use of cows to find whether more water causes higher milk yields?

28. Random placement of UNITS • This is an experimental technique for assigning subjects to different treatments (or treatment and control groups). • The thinking behind random assignment is that by randomizing who goes to which group, then the groups attributes will be roughly matched. • It goes back to the idea of keeping our subjects “as similar as I could have them” (in the words of James Lind) • Therefore any effect observed between treatment groups can be linked to the treatment, and is less likely to be caused by a characteristic of the individuals in the group.

29. Random assignment DOES NOT MAKE THE GROUPS REPRESENTATIVE! • It only makes sure the groups are matched. • If the units in your study have been randomly picked, they are representative, now using random assignment will keep them representative. • But if the units have not been collected by random sampling, you can use random assignment all you want, it’s not going to make them representative, it will just give you two matched unrepresentative groups http://www.bindichen.co.uk/

30. For Example: If I wanted to know if serif fonts cause easier reading in a wider population, this is how I would do it: http://www.bindichen.co.uk/ However this is not practical in year 12. Therefore we have to be very conservative about which wider population we can apply our findings to. So really, an experiment done in a year 12 class, even if it establishes causality, only applies to that class, or at most year 12s in our school Naturally you would wonder what is the point? Unfortunately, there is no other choice given the limited resources of a high school student. Hopefully you can see from the above diagram that with a little bit of money and time, powerful findings can come from the experimental process you are learning about right now.

31. How to do random placement • Step 1: Begin with a collection of subjects. For example, 28 students. • Step 2: Devise a method to randomize that is purely mechanical (e.g. flip a coin) • Step 3: Assign subjects with "Heads" to one group: Control Group Assign subjects with "Tails" to the other group: TreatmentGroup the one without the “treatment” the one getting the “treatment”

32. Does it matter that the groups have different numbers in them? NO NO NO NO NO!!!

33. “as similar as I could have them” • Random placement is great mostly for evening out demographics and characteristics between the groups.Eg. Same numbers of young and old people, male and female, Indians and White people, etc. • But there are other factors. Such as the time of day (reaction times are different just after you wake then when you are into your day). It is your responsibility to try to control these.