Disclaimer

1. Disclaimer • All workshops and workshop materials are the sole property of PEGS and cannot be published, copied, or disseminated without prior written approval from PEGS and are for student and faculty use only.

2. Assessing Research Sources Questions, Comments or Concerns? Contact us: (310) 243-2700or pegs@csudh.edu

3. 8 Criteria for Assessing Research Sources Author - Publisher - Currency - Purpose Objectivity - Argument - Accuracy - Tone • The first four are about understanding your research materials before you begin to read. - For more information about how to find and access reliable research materials check out our workshop “Library Navigation: Finding Sources Worth Keeping.” • The final four are about the evaluating the text while you read, and are the subject of this workshop. - Graduate level reading is critical reading. - By reading a single source critically, you enhance your understanding not only of that text, but of how it fits into the larger academic conversation.

5. The Purpose of ReadingWhat on earth are graduate students doing all this reading for?

6. Purpose: Why Research Is Important Each academic discipline has its own system of thought and methodology for gathering and processing information. A a graduate student, your job is to learn the methodology of your field of study and familiarize yourself with what others have said on relevant topics. Research demonstrates that you have sought out and considered a variety of viewpoints on a topic to strengthen and contextualize your argument. Quotes and citations provide evidence for your claims and give credibilityto your work.

7. Purpose: Join the Conversation “Academia is an on-going conversation, or a series of ongoing conversations, with the latest academics and scholars responding to each other and to the academics and thinkers of previous generations.” - Paul & Elder Professionals and academics join the [asynchronous] conversation by writing papers. The best research is not wholly original but develops from previous discoveries. When you read, listen for places where you find silences orgapsin the author’s argument. These are ideal points at which you can begin thinking about a paper topic.

8. Silences and Gaps • An author’s lack of objectivityor oversightmay offer you an opportunity to add to the body of knowledge being studied. • Write down your assessment of the author’s argument and specify what parts will be useful for your research and what the argument lacks that you will need to research further or even challenge. Click here for notes on the purpose of reading.

9. Before Reading: Know Your Source Author - Publisher • This workshop assumes that you are knowledgeable about who wrote and published the source, and how the text is generally treated/viewed by scholars. - Familiarity with your field develops over time. - Academic schools are smaller communities than you think; find the ‘big names’ and see who they thought was worth citing. - Check out PEGS Workshop “Library Navigation: Finding Sources Worth Keeping” for more detailed information on finding and vetting research sources. “Prewriting Strategies” also has a lot of useful tips on how to navigate research quickly to find those sources worth a closer look.

10. Know Your Source: A Quick Review Purpose - Currency Know the date your text was produced, and what type of publication it is. • Use peer-reviewed articles to support your research. Not all information is peer-reviewed. You may encounter… Popular Texts: mass-audience, non-specific Professional/Trade Materials: often specific, easy to read Scholarly Research: specific, technical, often requires knowledge of subject, rigorously cites sources in the form of footnotes or bibliographies • Your research will fall under three types of sources: Primary Sources: original, uninterpreted materials, historical [evidence] (i.e. artifacts, interviews, photographs, official records, statistics, newspaper articles, works of art, music, literature, any work that represents original thinking, et cetera) Secondary Sources: interpret/make claims using primary source information (i.e. journal articles, essays, bibliographies, commentaries, criticisms, histories, etc.) Tertiary sources: may cite, evaluate, collect, or distill primary as well as secondary sources (i.e. chronologies, directories, fact books, guidebooks, indexes, textbooks, manuals, and encyclopedias)

11. Active Reading

12. Active Reading Strategy: SQ3R - portions of SQ3R section based upon Landsberger, Joe. “SQ3R Reading Method.” Study Guides and Strategies. <http://www.studygs.net/texred2.htm> (Based upon Robinson, Francis Pleasant. (1970) Effective study. New York: Harper & Row. ) “SQ3R: Studying More Effectively.” Mindtools.com. <http://www.mindtools.com/pages/article/newISS_02.htm> Survey Question Read Recite Review

13. SQ3R: Surveying(What to Read Before you Read) Click here for notes on the process of surveying. Structural Reading: SQ3R –Survey, Question, Read, Recite, Review • Surveying: Scan & Skim the text, targeting and gathering together essential information by: • Read the title • Note each boldface heading and subheading • Note any charts, diagrams, or graphics • Note any reading aids • Read the introduction & conclusion, and summary &/abstract • Check out the list of references used by the author

14. SQ3R: Questioning(What to DO Before, During, & After you Read) Structural Reading: SQ3R –Survey, Question, Read, Recite, Review • Questioning: Using the information you gathered during your survey, ask questions of each heading, sub-heading, graphic, and chart (etc.) you encounter. You can also use this opportunity to ask general questions about the text. • Start by asking yourself what you know about each heading and sub-heading • Convert headings and sub-headings into questions • Write your questions down • Be a kid: Ask “why” and “how” a lot.

15. Surveying & QuestioningClose Reading Skill = Analysis Analyzing = to separate a whole into its parts. We start by looking at the parts the author has broken the paper into (surveying phase), then we ask questions to further break it down. • What, When, Where and Who Questions are often about determining meaning: • What’s the argument/main point? Whatconcepts, theories, or ideas does the author reference/use? • Where is the paper in the ongoing academic conversation? • Whois the author’s audience? Who/what is the paper responding to or addressing? • Why and how questions are often related to the author's purpose: • How does the text contribute to its field? How is heading 1 connected to heading 2? How does this statement relate to the author’s thesis? • Why did the author use this piece of evidence here? Why did the author change direction? Why is this text influential?

16. SQ3R: Reading & ‘Reciting’ (Delving Into The Text) Structural Reading: SQ3R – Survey, Question, Read,Recite, Review • Reading & ‘Reciting’: Methodical reading of the text, keeping survey information and questions in mind while taking notes. • Read with pen/pencil in hand, but concentrate on getting a sense of the main idea • Take notes & make annotations • Summarize pertinent sections • Highlight anything that strikes you as interesting or important • Expect to read a text multiple times (at least 3), especially if you’ll be using it for a paper or a test

17. SQ3R: Reading Tips • Answer the questions you asked as you surveyed the text & formulate more questions as you continue reading • Answer questions posed by your professor, by other students in class discussion(s), or in study guides • Look at examples (citations, graphs/charts, images) and make your own observations about them, then compare your observations to the author’s comments • Anticipate and predict changes/movements in the argument • Read & re-read difficult sections, paragraphs, or sentences slowly • After you read each section, immediately write down your reaction. You can say anything that comes to mind: good, bad or indifferent.

18. Reading & RecitingClose Reading Skill = Paraphrasing Example: “A man who stands for nothing will fall for anything.” -- Malcolm X • Understand what the writer said: Standing for nothing means believing in nothing. Falling for anything means being easily duped. • Express the author’s idea in your own words, restating the main point in one sentence: People who do not have strong convictions of their own can easily be tricked or led astray (Malcolm X).* • Elaborate that main point more fully in your own words. • Use metaphors, analogies, pictures, or diagrams to connect the writer’s main point to your prior knowledge of the subject.

19. SQ3R: Tips on Annotating Click here for notes on annotating. • Develop a consistent system for taking notes/annotating • Write down what you think • make predictions • ask questions • make general comments • make connections • summarize each paragraph or section • As with questioning, which kinds of notes will be useful to you depends on the purpose of reading

20. Reading & RecitingClose Reading Skill = Evaluating Evaluating = to consider the merits of something, to judge. When responding to a text, it is not enough to understand the material or what the author means to say, we must push forward and decide whether each part of the argument is valid, partially flawed, or faulty. We do so by assessing a text’s… • At the graduate level, evaluation is not about saying whether something is good or bad, but making careful observations • You cannot assess the final 5 criteria without striving to be objective yourself • Remember primary, secondary, and tertiary sources are evaluated differently: evaluation of historical documents, for example, is less about determining whether or not they are reliable than about what we can take from them Click here for notes on evaluating. Objectivity - Tone - Argument - Accuracy

21. Evaluating Objectivity No work is free from bias, so look for places where the author displays a particular bias. When the author is advancing an idea or conclusion, does he cite evidence to support his claim, or does he merely advance an opinion without support? Does the information express a specific agenda by selectively presenting or manipulating evidence? Every interpretation is influenced by the author’s context. Does the author use generalizations instead of specifics?

22. Evaluating Tone Most academic authors try to appear impartial, but some rely on rhetorical style to fill holes in a weak argument (this can also be a sign of bias): • Strong and impassioned language may indicate the author is too emotionally connected to provide an objective analysis. • Check for superfluous (too many, unnecessary) adjectives • Watch out for adverbs (‘ly’ words such as surprisingly, extremely, simply, etc.), which are often used to express authorial opinions in otherwise factual or objective sentences • Does the author try to appealto the audience’s sympathies or common knowledge? • Rhetorical device can stand in place of logical argument.

23. Evaluating Tone: Rhetorical Device • How many rhetorical devices does the author use? Do they occur at important parts of the argument? • Common rhetorical devices: analogy, colloquialism, & irony • Look for poetic language – statements that sound, or ‘ring’ true, but may not stand up to scrutiny. Two signs of poetic language are reversals & repetition: • chiasmus: reversal of terms in parallel clauses Ex: “Has the Church failed mankind, or has mankind failed the Church?”-- T. S. Eliot • anaphora: paced repetition of words/phrases Martin Luther King Jr.’s repetition of ‘I have a dream’ is an example of anaphora “All forms of the state have democracy for their truth, and for that reason are false to the extent that they are not democracy.” - - Karl Marx Wondering what that Karl Marx quote is all about? Click here.

24. Evaluating Argument Argument is about how the author uses evidence to support claims. Lack of objectivity or an emotional tone are signs of a flawed argument, so are “silences” and “gaps”in too great a number. • Is the argument persuasive? Why? (tone/rhetoric is not a good enough reason) • Are there parts of the argument that you find weak or not well- supported? (Does the author use examples to support claims?) • Is the argument logical or does the author take shortcuts? • 2 Great Sources for Info on Logical Fallacies: Purdue OWL & “Top 20 Logical Fallacies” from The Skeptic’s Guide to the Universe

25. Evaluating Accuracy • External Validity – the extent to which a study's results can be generalized or applied to other people/settings • Low external validity = results that cannot be reproduced outside the lab/setting. Poorly-conducted studies produce chance/coincidental results. • Internal Validity – how well a study was run/ how confidently one can conclude that the observed effects were produced by the independent variable and not extraneous ones. • Ask yourself: “Does the research prove what it says it’s proving?” "Was it really the treatment that caused the difference between the subjects in the control and experimental groups?" • Reliability - consistency of a set of measurements/measuring instrument, often used to describe a test • Can the research be repeated under the same conditions?

26. Target 1: The trial samples (black dots) are hitting close to the same mark = each piece of evidence reaches the same conclusion = study is reliable. However, the black dots are not centered near the bull’s-eye, thereby not achieving the correct conclusion. Study is not valid. • Target 2: Black dots are scattered above the target = each sample hits a different, but localized conclusion = meaning low reliability. The black dots are all shooting above bull's-eye meaning a conclusion other than the correct one is being found. So, study is not valid. • Target 3: The black dots are all over = study not repeatedly hitting the same outcome = each sample points to a different conclusion = study is not reliable. The farther each sample is from the bull’s-eye, the farther each outcome is from the study’s findings. So, the black dots are around the bull's-eye without hitting it =study has low validity. • Target 4: Each sample’s outcome is hitting the stated findings and doing so consistently = study is both reliable and valid. Reliable Not Valid Low Reliability Not Valid Not Reliable Low Validity Both Reliable And Valid - adapted from Experiment-Resources.com

27. SQ3R: Reviewing (Delving Into Your Notes) Structural Reading: SQ3R – Survey, Question, Read, Recite, Review • Reviewing: You didn’t do all the work of questioning, summarizing and annotating for nothing! Be sure to look back over what you’ve done. • Re-survey: go back over structural components of the text to remind yourself of the total article/study • Re-read sections of the text you highlighted/underlined • Re-read your own comments/questions • Are your questions still relevant to you? Do you have further questions? • Do you agree with your own comments, or has time given you a different perspective? • Write down further impressions • Look over your own notes more than once

28. Examples

29. Example: What is the author’s objective? What kind of organization would have published this? Is this a primary, secondary, or tertiary source? What can you say about its objectivity , tone, argument, and accuracy?

30. Citation: Lomborg, Bjorn. The Skeptical Environmentalist: Measuring the Real State of the World. Cambridge, New York: Cambridge UP, 2001. forest death 25% no forest death forest death from local pollution hard to study local pollution possible to regulate – SO2 no forest death due to SO2 counter measurement – no forest death trees growing faster (aided by N pollution) no forest death mis-measurement persistent report of forest death

31. Sample Argument #1 “In this new era, the single most immediate and most serious challenge to America’s traditional identity comes from the immense and continuing immigration from Latin America, especially from Mexico, and the fertility rates of these immigrants compared to black and white American natives. Americans like to boast of their past success in assimilating millions of immigrants into their society, culture, and politics. But Americans have tended to generalize about immigrants without distinguishing among them and have focused on the economic costs and benefits of immigration, ignoring its social and cultural consequences. As a result, they have overlooked the unique characteristics and problems posed by contemporary Hispanic immigration. The extent and nature of this immigration differ fundamentally from those of previous immigration, and the assimilation successes of the past are unlikely to be duplicated with the contemporary flood of immigrants from Latin America. This reality poses a fundamental question: Will the United States remain a country with a single national language and a core Anglo-Protestant culture?” Samuel P. Huntington, 2004 Member of Harvard’s Dept. of Government, 1950-1959 Associate Professor of Government at Columbia University, 1959-1962 Tenured Professor of Political Science at Harvard, 1963 - 2008

32. Sample Argument #1 Questions What is Huntington’s main argument? Is Huntington’s argument logical? Is it persuasive? Why? Why not? Does Huntington offer any proof of the claims that he makes in this paragraph? Does the author make claims without offering evidence? How many? Can you find weaknesses or gapsin his argument? Can you think of counterarguments that Huntington has not considered?

33. Sample Argument #2 “Much has been written on how the personal characteristics and working habits of fieldworkers mediate the cultural scenes that unfold in their presence. Women (or men) in the filed, for example, find some doors open more readily than others (Golde, 1970; Warren and Rasmussen, 1977). Rapport with certain informants may preclude it with other (Berreman,1962). Fieldworkers in some settings are granted relatively rapid access to culturally sacred matters; in other settings they will learn nothing about them unless they devote their professional careers to such a pursuit (Clifford, 1983b). John VanMaanen, Tales of the Field, U of Chicago P, 1988

34. Sample Argument #2 Questions What is VanMaanen’smain argument? Is the argument logical? Is it persuasive? Why? Why not? Does VanMaanenoffer any proof of the claims that he makes in this paragraph? Does the author make claims withoutoffering evidence? Can you find weaknesses or gapsin his argument? Can you think of counterarguments that VanMaanenhas not considered?

35. Special Thanks UNC Chapel Hill UC Berkeley Cornell University UNC Wilmington University of British Colombia MLA Purdue OWL Extra Credits Study Guides and Strategies Online And of course…… The Librarians at CSUDH LFC Cain Library

36. Need further help? • Check out PEGS! • Promoting Excellence in Graduate Studies • Make an appointment • (310) 243-2700 • pegs@csudh.edu • www.pegs4grads.org • Library Reference Desk • (310) 243-3586 or (310) 243-3582 • http://library.csudh.edu/services/reference/desk.shtml