How to Review and Evaluate a Scientific Article

1. How to Review and Evaluatea Scientific Article AEE Graduate Student Seminar Series January 16, 2002 Revised: February 5, 2003 Mikel Woods, Ph.D.

2. The Basics of a Critique • What is the specific question (s) being addressed? • What general research methods are used? • List the major findings of the paper and identify the figures and tables (data) which support those findings. • Are you confident about the data presented? If not, why not? What might be wrong? • What major conclusions do the authors make in their discussion? Are their conclusions supported by the data? How are these data important to the field (That is, what do these data tell us which is new?)? • Are there other possible interpretations (explanations) for the data? What are they? Girden, E.R. (1996). Evaluating research articles from start to finish. One Thousand Oaks, CA.: Sage Press.

3. Reviewing the Purpose • What were the objectives in performing the study? • What were the research questions being asked?

4. Study Design • What type of study was performed? • How was the study population sampled? • Are there potential sources of selection bias?

5. Measurement and Observation • Are clear definitions of terms used? • What outcome criteria were used? • What measurements were taken? • How? • What methods were used to assess validity or reliability of measurements?

6. Results • How are the data presented? • Is the data relevant to the study question? • Are there data that were not presented?

7. Conclusions • What were the main conclusions? • Is the study significant? • Is the study relevant to other populations? • What questions remain unresolved?

8. Reading the Fun Way • First thing to do is look at the four corners of the picture. (That way you don't miss anything.) • Then look at the center and find what catches your eye, • Then the foreground and background. • Then ask yourself, what is going on here?

9. Evaluating Your Peers What to Look For & How to Say it Constructively

10. Writing Evaluation Grid • Content • Organization, Rhetorical & Logical Development • Organization Sentence Structure • Word Choice • Tone • Spelling • Format

11. Use of Supporting Evidence • This point is underdeveloped. • The details here don’t really help me see your point. • I’m not convinced – what’s your authority? • The X (three) reasons you offer are very persuasive. • Good Examples. Can you offer more?

12. Use of sources. • Only two sources? You need several more. • Who said this? • One of the clearest paraphrases I’ve seen of this crucial passage. • Your summary leaves out three of the writer’s main points. • Your summary is just repletion – it doesn’t add anything new. • This quotation beautifully sums up your argument. • Why do you quote at such length here? Why not paraphrase? • You cite only sources that support your claim – citing one or two with differing views would help show me you’ve considered other opinions.

13. The Purpose • Why are you telling us all this? • What is the issue here, and what is your stand on it? • Very efficient and thorough discussion! You explain the content very clearly and thus reveal your understanding of the issue/topic?

14. Attention to Audience • Careful you don’t talk down to your readers. • You’ve left me behind here. I can’t follow. • Your level of jargon is perfect for relating to the audience. • I’m really enjoying reading this! • Don’t assume everyone shares your opinion about this issue.

15. Overall Impression • I was looking for more critical analysis from you, and I’ve found it! • Your grasp of the material here is truly impressive. • What happened here? I can’t understand your point in this article/section. • Good job -- you convinced me!

16. Overall Originations • I’m confused here – what does this point have to do with the one before it? • Your most important point is buried here in the middle. Why not move it up front. • Organization is chronological rather than topical; as a result, you summarize but do not analyze. • How did we get here? You need a transition.

17. Sentence Structure & Style • The passing of your sentences here really keeps me reading – excellent variation of length and type. • Can you combine sentences to make a logical connection explicit here? • Your use of questions helps clarify this complex issue. • This is not effective word order for a closing sentence – I’ve forgotten your main point. Can you find a better sentence? • These sentences all begin with nouns – the result is a kind of dull clip-clop, clip-clop.

18. Paragraph Structure • Why the one- and two-sentences paragraphs? Elaborate! • Your introductory paragraph immediately gets my attention and gives an overview of the essay – good! • I can’t follow the information in this paragraph. Can you reorder it? • What is the main idea of this paragraph? • Very effective ordering of details in this paragraph. • This paragraph skips around two or three points. It has enough ideas for three paragraphs.

19. Format • You need a title, one that gets across your meaning. • Your headings helped me follow this report. Why not use subheadings?

20. Documentation • I checked that source, and this quotation’s not on the page you list. • What are you paraphrasing here? Your introduction merely drops readers into the middle of things. Introduce the material paraphrased. • What are you summarizing here? Where do these ideas come from? • I can’t tell where this quotation ends. • Keep in-text citations as simple as possible – see information in style manual. • Why aren’t works listed in alphabetical order? • This is not correct APA citation style. Check your manual! • What is the date of this publication?