Chapter 8

1. Chapter 8 How good is the evidence: Personal Observation, Research Studies, and Analogies?

2. Personal Observation As evidence • Observations do not give us “pure” observations • What we “see” and report are filtered through a set of values, biases, attitudes and expectations • There is a need for you to determine whether there are good reasons to rely on such reports • The most reliable reports will be based on recent observations made by several people observing under optimal conditions who have no apparent strong expectations or biases related to the event being observed

3. Research studies as evidence • Research Studies • Form of authority that relies on observation and carries special weight • How dependable are the findings? • We turn to the scientific method as an important guide for determining facts because the relationships among events in our world are very complex and because humans are fallible in their observations and theories about these events So what does the scientific method do & what are it’s characteristics?

4. The scientific method • Attempts to avoid many of the build-in biases in our observations of the world and in our intuition and common sense • How does it do this? The following are the characteristics of the scientific method that make it unique and special …

5. The scientific method characteristics • Publically Verifiable Data • the scientific method seeks info in the form of publically verifiable data- that is – data obtained under conditions such that other qualified people can make similar observations and get the same results (replicate) • Control – the use of special procedures to reduce error in observations and in the interpretation of research findings (i.e. controlling bias – double blind?) • Precision in Language – precise and consistent in it’s use of language (use of operational definitions) When conducted well, scientific research is one of our best sources of evident because it emphasizes verifiability, control and precision

6. Research findings are sometimes undependable When communicators appeal to research as a source of evidence, you should remember the following: • Research Varies Greatly in Quality (research, like anything, varies from good to bad) • Research Findings Contradict One Another (go with those that have been replicated) • Research Findings Do Not Prove Conclusions (at best they support conclusions because researchers interpret the results & different people can interpret different ways)

7. Research findings are sometimes undependable 4. Regardless of how objective a scientific report may seem, important subjective elements are always involved • Researchers have expectations, attitudes, values, and needs that bias the questions they ask- the way they conduct research etc. 5. Writers often distort or simplify research conclusions

8. Research findings are sometimes undependable 6. Research “facts” change over time (once accepted – now refuted) 7. Research is sometimes more artificial then we think; in order to achieve control, research often looses it’s real world quality 8. The need for financial gain, status, security, and other factors can affect research outcomes (remember; researchers are just PEOPLE)

9. Research findings are sometimes undependable

10. Impossible certainty fallacy • When critically evaluating research conclusions, be wary of the reasoning error of demanding certainty in some conclusion when some uncertainty is to be expected but that does not negate the conclusion Fallacy Impossible Certainty • Assuming that a research conclusion should be rejected if it is not absolutely certain

11. Clues for evaluating research studies • What is the quality of the source of the report? • Does the report detail any special strengths of the research? • How recently was the research conducted, and are there any reasons to believe that the findings might have changed over time? • Have the study’s findings been replicated by other studies? • How selective has the communicator been in choosing studies?

12. Clues for evaluating research studies 6. Is there any evidence of strong-sense critical thinking? 7. Is there any reason for someone to have distorted the research? 8. Are conditions in the research artificial and therefore distorted? 9. How far can we generalize, given the research sample? 10. Are there any biases or distortions in the surveys, questionnaires, ratings, or other measurements that the researcher uses?

13. Generalizing from the research sample • Sampling – the process of selecting events or persons to study Why sample? • Because researchers can never study all people and events about which they want to generalize, they choose a sample – You need to keep several important considerations in mind when evaluating a research sample

14. evaluating a research sample • Sample must be large enough to justify the generalization or conclusion • The sample must possess as much breadth, or diversity, as the types of events about which conclusions are drawn • The more random the sample, the better (equal chance of sampling) Note: we can generalize only to people and events that are like those that we have studies in the research

15. Watch surveys and questionnaires • You can not assume that survey responses accurately reflect true attitudes • Ask yourself the following: • Were the questions ambiguous in their nature? (How were they worded?) • Were they answered honestly?

16. Analogies as evidence • Analogies involve comparisons and rely on resemblance as a major form of evidence • I.E. “We know a lot about something in our world (X), and another event of interest in one or more respects, then they will probably be alike in other respects as well.” • Argument by Analogy- an argument that uses a well-known similarity between two things as the basis for a conclusion about a relatively unknown characteristic of one of those things

17. To evaluate analogies focus on two factors • The ways the two things being compared are similar and different • The relevance of the similarities and the differences • Strong analogies will be the ones in which the two things we compare possess relevant similarities and lack relevant differences • Relevant similarities and differences are the ones that directly relate to the underlying principle illustrated by the analogy

18. Generate your own analogies • Identify some important features of what you are studying • Try to identify other situations with which you are familiar that have some similar features. Brainstorm – Try to imagine diverse situations • Try to determine whether the familiar situation can provide you with some insights about the unfamiliar situation

19. Fallacy: faulty analogy • The fallacy of the faulty analogy occurs when an analogy is proposed in which there are important relevant dissimilarities • In short, analogies that trick or deceive us fit our definition of a reasoning fallacy; such deception is call the faulty analogy fallacy