The scientific method

1. The scientific method F. Guesdon MED610 DDP March 2013

2. “Checker shadow illusion”, first described by Adelson, 1995 Is square A darker than B?

3. The “scientific method” model • Describes “best practice” method for scientific discovery • Developed from observation of succesful scientists

4. History 10th Century • Ibn al-Haytham (Alhacen): Pioneer in experimental optics and psychology, use of scientific method. 13th and 14th Centuries • Bacon (Collection of facts, induction) • Occam (Parsimony) 17th century • Descartes: Deductive method, predictions • Galileo: Experimental approach

5. History - Modern 20th century • Statistical criteria • Popper: Falsification • Kuhn: non-rational aspects

6. Our learning aims • Reflect on what makes research scientifically sound • Understand what the “Scientific Method” is • Ask if “Scientific Method” really accounts for all scientists need to do

7. Session plan Problem-solving strategies Case study 1: The law of falling bodies Case study 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

8. Thinking about probabilities • 1% of women have breast cancer (p = 0.01) • If a woman has breast cancer, the probability of a mammogram detecting it is p = 0.8 • If a woman has no breast cancer, the probability of the test being positive is p = 0.1 Estimate the probability that a woman whose mammogram came up positive actually has cancer From Gigerenzer, in The evolution of the mind, Dellarosa Cummins and Allen, Eds, 1988, chapter 1

9. Probabilistically correct answer • For every 1,000 women tested, 10 will have breast cancer and 990 won’t • Of the 10 women with breast cancer, 8 will be diagnosed correctly by the mammogram • Of the 990 other women, 99 will have falsely positive mammograms • For every 1,000 women tested, 99 + 8 = 107 mammograms will be positive • The probability that a positive mammogram indicates a true breast cancer is 8 / 107 = 7.47 %

10. There are two different ways of thinking

11. One way thoughts come to mind: Is happy This way of thinking uses perception and intution

12. Is angry This way of thinking uses perception and intution

13. Another way thoughts come to mind • Probability of having breast cancer is pc = 0.01 so of 1,000 women, we can expect 1,000 x 0.01 = 10 cases • And therefore 1,000-10 = 990 healthy women • The test has a rate of detection of pd = 0.8, so it should pick up 0.8 x 10 = 8 cases from the sample of 1,000 • The test has a false positive rate of p+ = 0.1, so 990 x 0.1 = 99 healthy women will also have a positive result • So, there will be in total 99 + 8 = 107 positive results. • If I am one of those, the probablility I have cancer is p= 8/107 ≈ 0.075

14. Type 1 thinking = “natural” • Automatic / intutive / effortless • Uses perception, common sense, training (skills) • Jumps to conclusion • “Heuristic”

15. Limitations of type 1 thinking • Perception (sensory) biases • We tend to misjudge numerical information • We tend to confuse the most typical with the most probable • We seek solutions that conform with how we perceive a problem rather than how it is objectively (framing, economy of thought)

16. Type 2 thinking: organised • Based on conscious processing • Rational, analytical • “Unatural”, difficult • Technically accurate • Slow or unconclusive when dealing with complex problems (social / economy etc.)

17. Common sense relies mostly on type 1 thinking • Provides fast, practical answers • Good for practical problems (hunting, farming, buying and selling, stay safe etc.) • Easy to commmunicate or convince – “Feels right” • Influenced by and produces common knowledge

18. Contemporary common knowledge • In a given situation, people from different cultures are likley to react differently • In a given situation, people will react differently depending on their personality traits • Women have better verbal skills and more empathy than men

19. Selecting candidates: Common sense approach • Candidates for PhD position selected by interview • Staff believe they select the best candidates • But they can only judge the performance of students they took in • So how do staff knowthat they select correctly? • We think common sense works because it “seems” to work

20. Science developped by mistrusting common sense and organising knowledge

21. Theory Hypothesis • Broad scope • Accommodates alternative hypotheses • Designed to be inclusive: incorporates as many facts and explanations as possible in a unified framework • Will be abandonned if cannot generate good hypotheses, or when a better theory is built • Focused • Generates specific predictions • Designed to be tested rigorously • Will be rejected as soon as it fails a single test

22. Problem-solving strategies: Common sense v. rational thinking Case study 1: The law of falling bodies Case study 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

23. Aristotle’s description of the motion of falling bodies The speed of falling objects is proportional to their weights. H L L H

24. What happens if a light object (L) is tied to a heavy object (H)? • The falling speed of the tied objects should be intermediate between those that they would have individually. • When tied, the two objects (H+L) form a single object heavier than H, so should fall faster than H alone.

25. Aristotle’s description can lead to contradictory predictions • Limited predictive value • Can lead to alternative contradictory predictions • Does not explain what it tries to describe

26. Identifying the problem 1 – Galileo noted the logical inconsistency in Aristotle’s description 2 - Observed that falling objects appear to start slowly and then accelerate 3 – Looked for supporting evidence: dents in a cushion 4 – Seeked to measure how speed increased with time and describe the relation in a manner fully consistent with measurements

27. How Galileo may have generated his hypothesis He uses the most simple mathematical description of accelerated motion: The speed (V) increases in direct proportion to time (T) since the object was dropped: V =  T

28. How to test this? T = 1 D = α The equation V =  T leads to a prediction about distance fallen with time: T = 2 D = 4 x α The distance (D) increases in proportion to time squared (T2): D =  T2 T = 3 D = 9 x α

29. The rolling ball experiments (1603) • Problem: Free fall is too fast • Solution: Study balls rolling down an inclined beam D3 D1 D2 Gallileo assumedthat this motion followed the same law as free fall

30. Simulated Galileo data The data does not fit perfectly the prediction Does that means the hypothesis is wrong?

31. Replicating the experiment The data is never perfectly reproducible either Does that mean the experiment is not reliable?

32. The most important step when interpreting data is ask if the data is good enough to mean anything. • Many experiments do not give “yes” or “no” answers, just “maybe” answers

33. The differences are not significant, so the data supports the prediction that D =  T2 Simulated Galileo data

34. Value judgments: Interpreting data • A researcher must interpret their data - decide what it means. • Interpretation is informed by controls (standardisation), replicationand statistical analysis • But not fully objective, depends on assumptions • The interpretation can be contested by other scientists (peers)

35. The most important step when interpreting data is ask if the data is good enough to mean anything. • Many experiments do not give “yes” or “no” answers, just “maybe” answers

36. The Scientific Method • Observe phenomena • Develop a hypothesis (inductive thinking) • Derive predictions from the hypothesis (deductive thinking). • Test one prediction (experiment) • Interpret the results: are they consistent with the prediction? • If yes, the model passes the test; test another prediction • If no, the hypothesis is proven wrong (falsified); alter or discard hypothesis

37. Inductive reasoning • Imagines possible causes or mechanisms to explain the data • Based on recognition of patterns or trends • Can be intuitive, subjective • Error-prone: risks confusing correlation with causality • Essential to make good hypotheses

38. Standard model of Scientific Method Induction Hypothesis Data Deduction Prediction Test

39. Hypotheses are at the core of the scientific method • A hypothesis is an attempt at explaining • Testing a hypothesis is testing our understanding • understanding means being able to make predictions! • This distinguishes investigative science from descriptive science (mapping, cataloguing, sequencing)

40. Testing hypotheses: Falsification • Experiments must be designed so as to reveal if the hypothesis is wrong • Experiments set up to confirm hypothesis are not informative Karl Popper (1902-1994)

41. Testing to faslsify… How would you test the following hypothesis?

42. “All cards that have a vowel on one side have an even number on the other side” 4 U

43. A J 2 7 Testing the hypothesis • You have a sample of 4 cards: Which card(s) do you need to turn over to test the hypothesis? Write your choice(s) on a piece of paper

44. A J 2 7 Prediction: Available cards: “All cards that have a vowel on one side have an even number on the other side”

45. Would turning card A test the hypothesis? • What might we find if we turn over card A? • An even number • An odd number • If it is and odd number, we will have learned that the hypothesis is false

46. A J 2 7 Apply this reasoning to all available cards “All cards that have a vowel on one side have an even number on the other side”

47. Correct choices: Cards A and 7 • If you find an odd number on the other side of A, you will know that the hypothesis is wrong • If you find a vowel on the other side of 7, you will know that the hypothesis is wrong

48. Were our initial choices wrong?Why? • Card 2 is not informative: whether there is a vowel or consonant on the other side will tell you nothing about the hypothesis - but many people choose it • Most people choose card A but very few people choose card 7 - this shows a natural bias towards seeking confirmation, but ignores half the evidence available

49. Problem-solving strategies: Common sense v. rational thinking Case study 1: The law of falling bodies 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

50. Pre-1984 view of peptic ulcer • Erosions of the lining of the stomach or duodenum • Believed to be caused by overproduction of stomach acids • Thought to result from lifestyle factors (stress or excess absorption of spicy food) • Treatments were: avoiding lifestyle factors, neutralising stomach acidity or preventing acid secretion by severing nerves • Alleviated symptoms, did not cure the disease