The Scientific Method

The Scientific Method Simon Hughes and Miaoqi Zhu

What is Science? Origin: Comes from the Latin word “scienta” – to know Mirriam Webster: a: Knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through the scientific method b: such knowledge or such a system of knowledge concerned with the physical world and its phenomena : Natural Science

The Scientific Method The process by which scientists acquire a systematic and reliable body of knowledge about the world. Often described in 4 steps: • Characterizations\Observations. Definition of the problem from what is already known. This could take the form of observations that cannot be explained using existing scientific knowledge, or the identification of a gap in our current knowledge that needs filling. • Formulate a Hypothesis. Develop a testable hypothesis that could explain the problem and provide a solution. • Make Predictions. What logically follows if 3 is true? What would disprove 3? • Experiment. Design and conduct some experiments to test the consequences of your hypothesis stated in 3. Do the data support or reject the hypothesis?

Real-Life Example We use the scientific method in our daily lives, we just don’t know it . For example: your car breaks down: • Characterizations\Observations: Your car won’t start • Formulate a Hypothesis: Maybe my battery is dead (one of many possible explanations) – Often you want to start with the most likely hypothesis • Make Predictions: If my battery is dead, then my lights and car radio also won’t work • Experiment: Try the headlights. They don’t work. Try the radio. It doesn’t work. Thus it is likely my battery is dead Solution: Replace the battery. Often the results of the experiment lead to a solution to the problem, prompt further experiments. Once you answer one question, you are often left with many more unanswered questions.

Scientific Example – Discovery of DNA’s Structure Note that Watson and Crick didn’t discover DNA, rather they determined it’s double-helical structure • Characterizations\Observations : Significance of the gene was known, but the structure of DNA was not • Formulate a Hypothesis : Crick and Watson hypothesized that the gene had a helical structure • Make Predictions: Crick formulates the transform of a helix showing an X- shape • Experiment : (1953) Primed by Crick’s transformation, Watson sees an X shape in Photo-51 (an X-ray diffraction image of the DNA molecule) Class Exercise: An other example

The Scientific Method – Broken Down • Research an area of interest to you • Define the question • Gather information and resources (observe) • Form a hypothesis • Perform experiments and collect data • Analyze and Interpret data, and draw conclusions • Define future work that follows on from your experiments • Publish results • Retest (frequently done by other scientists) There is an iterative process inherent in this approach that goes from steps 3 to 7

1. Research an area of interest to you • Find an area of interest to you • Review the literature on that area • Focus on high quality sources of information: • What is the source? • Books • Journals • Conference Proceedings • Workshop papers • Websites of researchers • Is it recent? • How many citations does it have? • Who funds the research?

1. Research an area of interest to you • Follow the trail of references • Start with a well regarded and recent paper, preferably a review paper • Obtain and read the references of that paper • Read the references of those papers, and so (bearing in mind criteria above) • If you start with a reputable source, you can somewhat rely on them to have validated the quality of their own refences

2. Define the Question • What do you want to know? • What do you want to find out about your subject? • Clearly define the problem statement as a question that can be investigated. • Examples: • How does the amount of water effect plant growth? • What limits the efficiency of solar panels? • How fast does Mars orbit the Sun? • Are there gaps in the existing knowledge? • Are there areas that a certain technique has not been applied? • Has all of the major work been completed in this area? • Are there flaws \ limitations with the current approaches?

3. Gather information and resources (observe) • What do you need to know so you can solve your problem? • Where \ how can you find this information? • What phenomena or processes exist that have not been explained, or possibly even researched? • No-one even thought to ask the question ‘what is gravity’ before Newton • Have observations been made that do not align with the current hypothesis?

3. Gather information and resources (observe) • E.g. Scientists observed that the perihelion (edges of the elipse) of the planet mercury exhibits a precession (rotation about its axis) that does not agree with Newton’s laws of motion • This is one of the observations that led Einstein to his theory of relativity

4. Form a hypothesis • Now you have done your research, you develop a hypothesis • A hypothesis is a prediction of the answer to you question • – abductive reasoning (covered later) • A hypothesis is an educated guess at the answer to a problem • This hypothesis also needs to make testable predictions so that it is falsifiable and can be tested through experimentation • Examples: • If a plant does not get enough water it will die • A neural network will out perform a decision tree on the task of image classification

5. Perform experiments and collect data • How will you test your hypothesis? • What tests will answer your original question? • How much data do I need to collect? • How do I collect the data so that I can perform the analysis I want to do on it • What are the appropriate metrics to use?

6. Analyze the data • What does the data tell you? • What conclusions can you draw from the data? • What patterns are present in the data? • What techniques can you use to help understand and visualize the data • Table • Graphs • Diagrams • Are there any gaps in your data? • Are there limitations as to what it tells you (indicates potential future work)

7. Define future work • Did your results support or reject your hypothesis? • If they supported your hypothesis, do you: • Need to gather more data in support of your hypothesis? • Need to address shortcomings or limitations in your experimental design? • Answer questions that could not be answered in your original work • Have questions that your results raise that warrant further investigation ? • If your results rejected your hypothesis: • Are there alternate hypotheses that could explain the problem? • Do your results suggest alternative approaches or solutions? • Negative results are just as valid as positive results • If your results, or some of your results, were inconclusive, then do you: • Need to gather more data? • Need to apply different techniques for adequate verification?

8. Publish results • Are your results significant enough to publish in their own right, or do you want to include them as part of some more significant piece of work? • What is the most appropriate medium to present your results? • Journal? • Book? • Conference? • Workshop? • etc 9. Retest • Other scientists will likely want to verify your experimental results by replicating them, and continuing your work

Standing on the Shoulders of Giants- Sir Isaac Newton • Science is a cyclical process, and builds on what has gone before • Einstein’s theory of relativity built on Newtonian mechanics , it enhanced it, it did not refute it • One experiment normally leads to many more, as some questions are answered, more new questions emerge

Questions Beget More Questions • E.g. When atoms were discovered, it was thought that they were indivisible, and the most granular form of matter (atomos: Greek for indivisible) • We later discovered that atoms are formed of particles – protons, neutrons and electrons • We now know that protons and neutrons in turn are made up of sub-atomic particles, quarks and gluons • String theory posits that all sub-atomic particles are made up of strings of energy vibrating to form some cosmic symphony • “It’s turtles all the way down!” – Stephen Hawking

The Atom and Beyond - Illustrated

Logic • Logical reasoning is important to the scientific process. • To formulate hypotheses, make predictions from your hypotheses, and interpret and analyze data, you need to use logical reasoning skills • There are two main forms of logical reasoning: • Deductive logic • Inductive logic.

Deductive Reasoning • Starting with one or more premises, or axioms (self evident truths), deductive logic proceeds as a sequence of deductions • If A then B, if B then C, and so on. • “Elementary My Dear Watson” • For deductive reasoning to be sound, the conclusion must always follow from the premise. • If the premise is true, then the conclusion is true • The opposite case cannot be assured (a common fallacy ) • e.g. Every human that is alive has a beating heart. If something has a beating heart, it does not mean that it is human. • ‘Affirming the Consequent’

Deductive Reasoning • Deductive logic is important for conducting scientific research • We use deduction primarily to • Formulate hypotheses • Form predictions based on those hypotheses • Test theories (our own, or other’s) • For scientific results and conclusions to be valid, the reasoning process must be sound (i.e. logical) • Facts based on unsound or illogical reasoning cannot be relied upon, and should not be incorporated into the body of scientific knowledge

Deductive Reasoning - Illustrated

Inductive Reasoning • Inductive reasoning begins with an observation, not a pre-existing premise or assumption • In inductive reasoning, people draw conclusions about broad classes of objects or phenomena based on some more specific observations. • Scientists examine a sample and then draw broader conclusions about the population from which the sample was drawn. • In the scientific method, this approach often comes into play when making initial observations, and when analyzing data looking for patterns • The process involves formulating generalizations based on some observed patterns or principles • If A has property B, and A is a C, then other C’s also have the property B

Inductive Reasoning Illustrated General Principle Observation Observation Observation Observation Observation

Inductive Reasoning For example: Observations: • You drop a ball, it falls to the ground • You let go of a fork, it also falls to the ground • You observe a bike leaning against a railing fall to the ground Conclusion • You may then conclude that all objects with mass fall to the ground if unsupported, a force we call gravity

Inductive Reasoning • Suppose you bite into a number of small, green apples and they all taste sour • From that observation you would surmise that all small, green apples taste sour • (from http://www.scientificmethod.com ) • Or, as a child, you may notice that every dog you meet barks, so you surmise that all dogs bark

Deduction Versus Induction • Principles derived from inductive reasoning don’t come with the same guarantees as those from deductive logic • In a deductive propositions, the consequent always follows the antecedent • So “If A then B” implies A entails B, i.e. if A is true then B is always true • Inductive statements are generalizations, they state general principles that are commonly true, but not always • E.g. ”Things that go up come down” • This holds for most objects, but not for those travelling under their own locomotion, such as a plane or rocket. • Or “fruit generally tastes sweet” • This holds for strawberries, raspberries and so on, but not for lemons and grapefruits.

Abductive Reasoning • Abductive reasoning involves proposing potential causes for an observation, often selecting the most probable • Similar to deductive reasoning in reverse, i.e. given B, A is the most likely cause • Particularly prevalent in science, this is the reasoning process used to derive hypotheses • Normally there are many potential causes for a given event, abductive reasoning is providing an educated guess as to what they might be, and often selecting the most probable cause so you can test it

Deductive Reasoning A B Cause Effect Abductive Reasoning ? B Explanation Observation

Can Science Prove Anything? • We have this extensive, wide-reaching system of knowledge about the world acquired through the scientific method

Can Science Prove Anything? • We have this extensive, wide-reaching system of knowledge about the world acquired through the scientific method • Is any of it proven?

Can Science Prove Anything? • We have this extensive, wide-reaching system of knowledge about the world acquired through the scientific method • Is any of it proven? • Is there a proven scientific theory?

Can Science Prove Anything? • We have this extensive, wide-reaching system of knowledge about the world acquired through the scientific method • Is any of it proven? • Is there a proven scientific theory? • How do we know it’s true?

Can Science Prove Anything? • We have this extensive, wide-reaching system of knowledge about the world acquired through the scientific method • Is any of it proven? • Is there a proven scientific theory? • How do we know it’s true? • In science, we can never know anything for certain, we can only develop theories, and then test them in the real-world

The Nature of Proof • Science and mathematics differ in one fundamental way - you can definitively prove something from first principles using mathematics or formal logic • You can never prove anything definitively in science, you can only falsify, or disprove. • There are no “proven theorems”or facts in science. • This is why it is important to develop theorems that are falsifiable • As a consequence, a good scientific theorem is testable i.e. it makes predictions about the real world that can be tested (and disproven if wrong)

The Nature of Proof • We believe theorems such as evolution and relativity which have withstood extensive attack and scrutiny from the scientific community, and for which a substantial body of evidence has been accumulated • It can take years to amass evidence to fully support a complex theory • E.g. the global warming debate – ‘Has the activities of mankind led to global warming’ • In contrast, it only takes one piece of reliable evidence to disprove or refute a theory

The Nature of Proof • One of the principle purposes of the peer review process is to validate a theory by ensuring it can withstand scrutiny by the scientific community • Research that disproves existing theorems is just as valid and useful as research that develops new theorems. • Once a theorem has been disproved, science can proceed in other more fruitful directions. • Through this process of critical evaluation, we acquire the system of ‘reliable knowledge’ that we talked about during lectures 1 and 2

Null Hypothesis • As we can only refute theories, we cannot prove them, scientists often create a null hypothesis and try to disprove it • The null hypothesis is normally the opposite hypothesis to the one you believe is correct • By disproving the null hypothesis, you lend support to you original hypothesis • This uses a technique logicians call “proof by refutation”, or the ‘law of the excluded middle’ • A can only be true or false. • By proving Not A = false, then A = true • Failing to disprove the null hypothesis may indicate problems in your original hypothesis, or you just may need to take a different approach

Null Hypothesis • E.g. You may want to attempt to demonstrate that people learn more effectively when studying in the morning as opposed to in the evening • You would then state your null hypothesis as “there is no difference in how effectively people learn between studying in the morning or in the evening” • And then perform some experiments and gather some data to test this theory • Often very little data is needed to disprove a theory, but to ‘prove’ something, you need to gather a lot of data

The Scientific Method