Reading And Evaluating Research Reports

1. Reading And Evaluating Research Reports Dr. Belal Hijji, RN, PhD September 16, 2010

2. Read Chapter 26: Pp 655-672Polit, D. & Beck, C. (2004) Nursing Research: Principles and Methods. 7th edition

3. Learning Outcomes At the end of this lecture, students will: • Recognise the importance of critiquing a research report • Identify the elements to consider when evaluating research reports • Be able to critique the following aspects of a research paper: • The biographical details of the researcher(s) • The source(s) of funding • Details provided in the abstract • The aims of the study • The literature review 3

4. Please make sure to read the following research articles if you have not done so: • ALLEN, M. ET AL. Effectiveness of a preoperative teaching programme for cataract patients. Journal of advanced nursing, 17: 303–309 (1992). • MCBRIDE, A. Health promotion in hospitals: the attitudes, practices and beliefs of hospital nurses. Journal of advanced nursing, 20: 92–100 (1994). • KING, K.M. & JENSEN, L. Preserving the self: women having cardiac surgery. Heart and lung, 23(2): 99–105 (1994). 4

5. The Importance of Critical Evaluation • Not all research documents are of high quality; most research has limitations and weaknesses. Some studies are worthless while others are excellent. • Nursing practice can be based on robust evidence. • Nursing knowledge can be advanced. • For nurses, identifying both the strengths and weaknesses of a research document is useful in a number of ways. These are: • Identify good quality research documents related to question in hand • Recognise credible material that support or challenges insights gained through intuition, education, or experience. • Make informed decision about appropriate care

6. Factors For Critical Evaluation • The biographical details of the researcher(s) • The source(s) of funding • Details provided in the abstract • The aims of the study • The literature review • The choice of research methods • Pilot study • Sampling procedures • Data collection methods and data analysis • Presentation and discussion of the results • References • Ethical issues • Rigour and relevance to nursing practice

7. Biographical Details of the Researcher(s) • Providing details about the researchers’ qualifications, current posts and/or academic background is good research practice. It facilitates critical consideration of the claims made in the document. For example, a team of doctors researching nurse education may employ different research methods and/or come to different conclusions than a team of nurses or a mixed team of doctors and nurses. • All papers you have show that the authors have nursing qualifications, such as RN and RGN; some have academic qualifications such as PhD, MA and MN, suggesting that they had training in research methods and/or experience of carrying out research. Some of the authors’ current posts are also given. Only article 2 by McBride gives enough detail to enable us to see how the current post of the author relates to the subject of the research.

8. Source(s) of Funding Only the article by King & Jensen acknowledges a source of research funding. We do not know whether the research projects described by Allen et al. and McBride had any external funding. Nevertheless, there are no obvious ways in which the source of funding might have affected the conduct or findings of the three research projects. If, for example, the project described by Allen et al. had tested a commercially available teaching package, we might have wondered whether the study had been funded by the company producing the package or by a commercial rival

9. Details Provided in the Abstract • An abstract (short summary) is always included in articles published in good quality health journals such as the Journal of advanced nursing and the British medical journal. • Abstracts provide an accurate summary of the content of a paper and are composed of the aims, methods, findings, conclusions and recommendations. An abstract is useful because it: • Can often provide a quick assessment as to whether a document is relevant to your topic of search or your clinical practice. An abstract can therefore save time and, sometimes, money. • May provide indications that a document has serious methodological flaws, or that the authors have reached far-fetched conclusions. Such suspicions can then be confirmed or rejected by careful reading of the main body of the document. Continued…….

10. The abstract of the article by Allen et al. is extremely short. The aims of the research and sampling procedures are unclear. No mention of the research design and data collection methods. It suggests that the findings are unequivocal. • The abstract provided by McBride is much longer with a lot of detail provided about aims, sampling, data collection, findings and recommendations. It does not, however, give a specific name to the research design. • The abstract of the article by King & Jensen is the clearest of the three. This is partly because it is presented under separate headings. We are given clear details under the headings Objective, Design, Setting, Participants, Results and Conclusion (see next slide).

11. Preserving the self: women having cardiac surgery King KM, Jensen L. Abstract Objective: To explore the process that women undergo when having cardiac surgery. Design: A grounded theory (slide 12) approach was used to gain an understanding of the core process. Setting: A large hospital with an active cardiovascular surgery practice. Participants: Ten women who had undergone cardiac surgery in the last 3 months. Results: Behaviors in which the women were engaged were directed toward "preserving the self" while getting through the surgical experience. This process had five phases: waiting, getting there, surviving, being there, and moving on. Conclusion: By having a clearer understanding of the process women undergo when having cardiac surgery, health care providers will be able to participate more effectively with women through their experience.

12. Grounded Theory • The grounded theory was borrowed from sociology and employed more often by qualitative researchers. It is an approach to study social processes and social structures. • Qualitative researchers aim to develop a conceptualisation of the phenomena under study that is grounded in the observation they make. The primary purpose of the grounded theory is to generate comprehensive explanations of phenomena that are grounded in reality. • The focus of most grounded theory studies is the development and evolution of social experience – the social and psychological stages and phases that characterise an event or episode. • One of the fundamental feature of this theory is that data collection, data analysis, and sampling occur at the same time. • Data sources for this theory could be interviews, observations or existing documents.

13. Aims of the Study • The aims of the research should be made clear early in the document (usually in a section entitled Introduction or Background). • The aims may be stated in terms of objectives, research questions and/or hypotheses. Aims • The aims of a piece of research are often expressed in a broad statement of intent, such as a description of the factors that may influence clinical outcome and satisfaction in patients treated for urinary incontinence.

14. Objectives • Objectives break aims down into specific elements to determine: • what treatments patients have received for their incontinence; • whether each treatment was given by a doctor, nurse, physiotherapist or other worker; • the perceived effectiveness of each treatment; • patients’ satisfaction with each treatment; • factors contributing to satisfaction or lack of satisfaction with each treatment; and • the clinical outcome of and patients’ satisfaction with their overall treatment.

15. Research questions • The term “research question” is more often used to describe a specific question to be answered by the research. There may be a number of these questions for any one research project. The aims and objectives above could be expressed in terms of research questions of the following type. • What are the factors that influence clinical outcome and satisfaction in patients treated for urinary incontinence? • What treatments have patients received for their incontinence? • Was the treatment carried out by a doctor, nurse, physiotherapist or other worker?

16. Hypotheses • A hypothesis is a statement about the relationship between two or more variables. Hypotheses characterise quantitative, not qualitative, research. • A variable is any characteristic of a person, thing or situation that can vary. Age, weight, type of medication taken, smoking behaviour and beliefs are all variables. • A variable could be independent (presumed cause) or dependent is the “presumed effect”. In the study on incontinence described above, factors related to treatment would be IVs and clinical outcome and patient satisfaction would be DV. • Examples of research hypotheses are: • wounds heal more quickly by using lotion A than by using lotion B; and • cigarette smokers have higher rates of lung cancer than non-smokers. • The type of lotion and smoking behaviour are the IVs, while speed of wound healing and rates of lung cancer are the DVs. Certain variables may need to be defined. For example, when is a wound to be considered healed and who is classified as a smoker? If a hypothesis is to be subjected to statistical testing, a null hypothesis is sometimes formulated.

17. Exercise • Read the Introduction to Allen et al., the subsection marked Aims in McBride and the opening (untitled) section of King & Jensen. • How easily can you find the aims of each study and how clearly are the aims expressed? Are the aims stated in terms of objectives, research questions or hypotheses? • Make some notes on your answers to these questions and then read the relevant discussion in Section 6

18. Literature Review • The literature review allows researchers to set their research topic in the context of existing knowledge. As such, the their contribution to knowledge can be demonstrated. • The review appears in a specific section or forms part of an introduction. It should discuss material relevant to the aims of the research. • What’s to do if the literature search revealed insufficient (or no) relevant nursing literature? Be explicit and try to review parallel literature. • What’s to do if there are no relevant research documents from your country? Look for the international literature • Concentrate on reviewing primary, rather than, secondary sources of research. Why? (19) • Be cautious if the references provided in the review are more than five years old. Why? • A literature review should include documents published on a subject, and the ideas in different documents should be compared and contrasted. Gaps in certain areas of the literature should be acknowledged. (19) 18

19. In a paper on Turkish nurses’ knowledge and practice of blood transfusion, Bayraktar & Erdil (2000) claimed that Tylor et al. (1993) stated “central venous route is not appropriate [for blood transfusion]”. Further review of this reference revealed that rapid transfusion through a central line can result in cardiac arrhythmias but did not discourage using central lines. • According to Hijji (2007), patient preparation before blood pack collection is an aspect previous research studies did not evaluate.

20. Referencing Systems • The main systems used for citing and referencing other work are: • The Harvard system: • References are cited in the text by giving the name(s) of the author(s) followed by the year of publication. • Polit & Hungler(1989) pointed out that … • It has been observed (Briggs, 1972) that … • The numerical system. • References are numbered consecutively as they occur in the text. The names of the author(s) may or may not be given: • Polit & Hungler (11) pointed out that … • It has been observed (5) that …

21. Exercise 4 • Can you locate the literature review in each of the three articles? Do the reviews discuss literature directly relevant to the aims of the research? Are the documents reviewed evaluated in terms of quality? Are the ideas in different documents compared and contrasted? Are most of the references recent (from the last five years)? Do the three articles use numerical or Harvard referencing systems?

22. Read Chapter 3: Pp 46-60Polit, D. & Beck, C. (2004) Nursing Research: Principles and Methods. 7th edition

23. Choice of Research Methods • The aims should be restated at the start of the Methods section to help you assess whether the methods chosen by the researchers were appropriate to meet the aims of the study.

24. Quantitative or Qualitative Research? • The aims should be restated at the start of the Methods section to help you assess whether the methods chosen by the researchers were appropriate to meet the aims of the study.