Causality, Reasoning in Research, and Why Science is Hard

1. Causality, Reasoning in Research, and Why Science is Hard Sources: D. Jensen. “Research Methods for Empirical Computer Science.” William M.K. Trochim. “Research Methods Knowledgebase”

2. More on Causality • What is causality?

3. What’s Important About Causality? • Explanation • Association provides prediction, but not explanation • Identifying causal mechanisms may uncover underlying reasons for relationships • Control • Understanding causality allows us to predict the effects of actions without performing them • Allows more efficient exploration of the space of possible solutions

5. Conditions for Causal Inference

6. Problems with Association

7. Are Feathers Associated with Flight? Do they have a casual relationship with the ability to fly?

8. Related Fallacies • Common (Questionable) Cause Fallacy • This fallacy has the following general structure: • A and B are regularly associated (but no third, common cause is looked for). • Therefore A is the cause of B. • Called “Confusing Cause and Effect” fallacy, if in fact, there is not common cause for A and B • Post Hoc Fallacy • A Post Hoc is a fallacy with the following form: • A occurs before B. • Therefore A is the cause of B.

9. Eliminating Common Causes

10. Control

11. Randomization

12. Modeling

13. Reasoning Methodologies in Research

14. Types of Reasoning in Research

15. Deductive vs. Inductive Methodologies • Deductive • Inductive

16. What is Abduction?

17. Examples of Abductive Reasoning • A Medical Diagnosis • Given a specific set of symptoms, what is the diagnosis that would best explain most of them? • Jury Deliberations in a Criminal Case • Jurors must consider whether the prosecution or the defense has the best explanation to cover all of the evidence • No certainty about the verdict, since there may exist additional evidence that was not admitted in the case • Jurors make the best guess based on what they know

18. Abductive Reasoning in Science • Abduction selects from among the hypotheses being considered, the one that best explains the evidence • Note that this requires that we consider multiple alternative hypotheses • Abductive Reasoning is closely related to the statistical method of Maximum Likelihood Estimation • Possible threats to validity • Small hypothesis spaces • Small amounts of evidence to explain

19. Challenges in Abductive Reasoning • Creating hypothesis spaces likely to contain the “true” hypothesis • Approach: create large hypothesis spaces • Knowing when more valid hypotheses are missing from the hypothesis space • Approach: constantly evaluate and revise the hypothesis space (multiple working hypotheses) • Creating good sets of evidence to explain • Approach: seek diverse and independent evidence with which to evaluate hypotheses

20. Why use multiple working hypotheses? • Objectivity: Helps to separate you from your hypotheses; shift from personal investment in hypotheses to testing the hypotheses • Focus: Reinforces a focus in falsification rather than confirmation • Efficiency: Allows experiments to be designed to distinguish among competing hypotheses rather than evaluating a single one • Harmony: Limits the potential for professional conflict and rejection because all hypotheses are considered and evaluated

21. “Strong Inference” • John R. Platt, Science, October 1964 • “Strong Inference - Certain systematic methods of scientific thinking may produce much more rapid progress than others.” • Not all science/research is created equal • Don’t confuse research activity with effective research • Activity: building systems; proving theorems; conducting surveys; writing and publishing articles; giving talks; obtaining grants • Research: improved predictions; better understanding of relationships; improved control of computational artifacts • Many researchers are active; only a subset do effective research

22. Initial Questions for “Strong Inference”

23. Arguments and Fallacies • Aside from general reasoning methodologies, one must ensure the validity of all arguments used in any research endeavor • An argument • Consists of one or more premises and a conclusion • A premise is a statement (a sentence that is either true or false) that is offered in support of the claim being made, which is the conclusion (also a sentence that is either true or false) • Modus Ponens (and Modus Tollens) • A fallacy • Generally, an error in reasoning (differs from a factual error), • An "argument" in which a logically invalid inference is made (deductive) or the premises given for the conclusion do not provide the needed degree of support. (inductive)

24. Common Fallacies • Ad Hominem • Appeal to Authority • Appeal to Belief • Appeal to Common Practice • Appeal to Popularity • Begging the Question • Biased Sample • Hasty Generalization • Ignoring A Common Cause • Burden of Proof • Straw Man • See: • http://en.wikipedia.org/wiki/Fallacy • http://www.nizkor.org/features/fallacies/

25. Why is Science Hard?[Notes]