The Scientific Literature: Scientific Appraisal

1. The Scientific Literature:Scientific Appraisal John Bycroft Sarah Knight

2. Purpose of Scientific Literature • Communication between researchers • Historical record of progress of scientific research • Avoids duplication of results and methods • No Journal of Negative Results so some unsuccessful studies may be repeated.

3. Why Read the Scientific Literature? • Keeping up with current scientific / clinical findings • Evidence-based medicine decisions • Developing new methodologies • Find deficiencies in current knowledge • Preparation of a literature review for a thesis • Writing a review article • Reviewing a paper for a journal

4. Why is Critical Appraisal Important? • Determine the true results and conclusions • Assess the validity of the results • Assess the relative importance of the study

5. Types of Publication • Peer reviewed journals • Specialist (Urology, BJUI) • Populist (Nature, Science etc) • In-house publications • Conference proceedings • Peer reviewed • Non-peer-reviewed • Books • Theses

6. Types of Article • Original Article • Review Article • Technical Note • Letters to Editor • Short Communication • Case Report

7. The Scientific Method • Anecdote or Observation • Literature Review • Hypothesis • Design of experiment under controlled conditions to test hypothesis • Disproof or support • Publication • Hypothesis  Theorem

8. Anatomy of a Paper • Abstract • Introduction/Background • Methods & Materials • Patient Selection • Results • Discussion • Conclusion

9. Abstract • Summary of problem and main results • Beware of assuming results from reading the abstract alone. This may not have been written by the authors and may be misleading or not contain all useful findings of the study • You can’t appraise a paper from the abstract alone

10. Introduction • Understanding and presentation of previous work (by others and themselves) • Clear statement of problem and hypothesis • How they intend to answer problem or prove hypothesis

11. Methods & Materials • Detailed enough to allow repetition of study • Experimental Design • Statistical Tests • A priori • P value for null hypothesis • Repeatability and reproducibility

12. Experimental Design • Test and Controls • Sample size • power equation • Patient Selection • Inclusion/exclusion criteria • Randomisation • Matching (age/sex) • Animal model • Study Design • Longitudinal/Retrospective • Placebo controlled etc

13. Validation of Methods • Specificity, precision, reliability • Error of measurements • Subjective or objective • inter-, intra-rater variability • Bias • Diurnal or hormonal variations

14. Results • Appropriate presentation • Graphical • Tabular • Statistical Analyses • Factual statements about observations

15. Discussion • Brief summary of methods • Explanation of limitations • Implications of the results, individually and collectively • Reference to previous studies • Overall conclusion

16. Conclusions • Supported by experimental results • Objective • Answer the hypothesis originally stated

17. Key Points for Scientific Appraisal • What are you trying to achieve? • Read through completely • Find statement of problem • Find key results and conclusions • Determine strengths and weaknesses, e.g. major experimental design flaws • Are conclusions supported by results? • How are the conclusions or other findings relevant to you

18. Typical Journal Reviewers Checklist • Comments on the novelty of these findings • Do the findings of this study represent a major (rather than incremental) advance over previously published observations? • Is this study hypothesis driven? Courtesy of the Journal of Applied Physiology

19. Typical Journal Reviewers Checklist • Is the rationale for this study logically presented? • Are there major concerns regarding the experimental design and methods? • Are the results clearly presented? • Can the discussion be improved?