Reading And Evaluating Research Reports (Continuation)

1. Reading And Evaluating Research Reports(Continuation) Dr. Belal Hijji, RN, PhD September 30, 2010

2. Quantitative or Qualitative Research? • To study a topic, a decision is to be made whether the methods used are quantitative, qualitative, or both. • Quantitative research seeks to answer questions of “how much?” and “how many?” and is concerned with the relationships between variables. • Qualitative research is concerned with a matrix of “who, why, when and where?” questions and with the exploration of issues. • Experiments, quasi-experiments, and surveys, which generate numerical data for tabulation and/or statistical testing, are forms of quantitative research. • Qualitative research utilizes unstructured or semi-structured interviews, participant observation and/or the analysis of documents. Qualitative data are in the form of words, such as interview transcripts and researchers’ field notes. Data are assigned to thematical categories, which are used as a basis for discussion. (See example next slide) 2

3. Hijji (2007) asked participants in the focus groups to explain why the knowledge that nurses had was not applied in practice. One of the nurses said: "Limited resources in workplace, lack of staff is another important consideration and heavy nurse: patient ratio. Also being busy with other patients or increased pressure of work and lack of time [with which] we cannot concentrate very well to apply everything we know procedure wise". (ADM-1) • The above answer refers to ‘lack of resources’ as a descriptive category. Various descriptive categories such as ‘different conditions of employment’, ‘central decision making’, ‘lack of proper appraisal mechanism’, ‘lack of clinical supervision’, and ‘lack of planning and organisation’ could be grouped together under the theme of ‘ORGANISATIONAL BARRIERS’.

4. Positivism underpins quantitative research. Positivism is a philosophy based on ideas about the existence of objective reality and the ability of human beings to discover facts if they avoid personal biases and take a scientific approach. Positivist philosophy and quantitative method underpin most research into physical, chemical and biological experiments, medical research, and some psychological experiments and social surveys. • Interpretivism underpins qualitative research. Interpretivists assert that humans are not completely objective and that subjective interpretation is inevitable in order to construct knowledge. Interpretivists believe that human beings can still be scientific (systematic and rigorous), without wholly eliminating their biases. • Qualitative studies could be ethnographic (exploring social/cultural interaction and meanings) or phenomenological (focusing on individuals’ experiences). • Quantitative researchers are sometimes disparaging about qualitative research and vice versa. Some quantitative researchers describe qualitative research as unscientific, biased or anecdotal, and claim that it produces no “real knowledge”. • Some qualitative researchers believe that quantitative research seeks to reduce complex issues to simplistic explanations, fails to explore contextual matters and takes insufficient account of the effects of the researcher on the process and products of research.

5. Table 1. Key differences between quantitative and qualitative research

7. Which of the papers is quantitative or qualitative research? • ALLEN, M. ET AL. Effectiveness of a preoperative teaching programme for cataract patients. Journal of advanced nursing, 17: 303–309 (1992). • MCBRIDE, A. Health promotion in hospitals: the attitudes, practices and beliefs of hospital nurses. Journal of advanced nursing, 20: 92–100 (1994). • KING, K.M. & JENSEN, L. Preserving the self: women having cardiac surgery. Heart and lung, 23(2): 99–105 (1994).

8. Quantitative Research Forms • Experimental: Experiments test cause and effect relationships between variables, and if true, they have three characteristics: • Manipulation: Doing something (introducing experimental treatment) to at least one group of subject. (Slide 32) • Control: The experimenter must introduce one or more controls over the experimental situation. Control is acquired by manipulating, by randomising, by careful preparation of experimental protocols, and by using a control group. (Slide 33) • Randomisation. The experimenter must assign groups to treatments as well as subjects to a control or experimental group randomly. (Slide 34) Continued…

9. Quasi-experimental: A quasi-experiment lacks either randomization or a control group, due to contextual limitations.. For example, we may wish to determine the physical and psychological effects of a new shift-work system on nurses working in a particular hospital. Since the new shifts are being introduced for all nurses throughout the hospital, randomization is not possible. It may, however, still be possible to collect pre-test and post-test data and to use the nurses at a similar hospital, which is keeping the old shift system, as a comparison group. • Survey work: Surveys can be used to collect information about variables and, sometimes, to test relationships between variables. Information can be collected about people’s health status, living conditions, or health beliefs. The information is turned into numerical data for statistical analysis. • Survey data are obtained by means of questionnaires or using a highly structured interview schedule. Interviews may be conducted face-to-face or by telephone. • Some surveys are designed to gather information from the entire population of (for example) a country or all the patients who attend any health clinic. A survey of an entire population is known as a census, and may be costly and time-consuming to conduct. An acceptable alternative to this is to survey a sample and generalise the findings obtained from it to the target population.

10. A Research Hypothesis “The color of a mental health nurse’s uniform affects the degree to which psychiatric patients display positive affective behaviours (smiling, laughing)”. The independent variable is the uniform color which could be manipulated by assigning some nurses white uniforms and other nurses brightly colored uniforms. We can compare, 24 hour after admission, the affective behaviour (dependent variable) of both groups of patients; those who were cared for by nurses in white uniform and those cared for by nurses in colored uniform.

11. The Function of Control in Experimental Context • Obtaining evidence about a relationship between two variables requires at least one control group. For example: • Assume that we supplemented the diet of a group of pre-mature infants with a particular combination of vitamins and other nutrients every day for 2 weeks. Suppose that their pre-treatment average weight is 2 kg and after treatment is 2.5 kg. What would the weight of the infants after 2 weeks tell us about the effectiveness of the treatment? Nothing. • Can we conclude that that there is a causative relationship between the independent variable and the dependent variable? Why? Because infants normally gain weight as the mature. • Without a control group that receives no nutritional supplements, it is impossible to separate the effects of maturation from those of the treatment.

12. The Randomisation Procedure • You have a list of 15 children who are for tonsillectomy. We are interested in testing the effectiveness of two nursing interventions with regard to the child’s level of pre-operative anxiety. The interventions are: Procedural information, sensation information (what the child will feel). The third group will receive no special intervention. The list of children’s names is enclosed. Each child received a serial number. Using the attached table of random digits, perform a random assignment of children to groups, and groups to treatments. • Identify a starting point on the table • Move from that point in any direction, for example to the right • Select the first five numbers that fall between 01 and 15 • Assign the first five numbers to group 1 and so forth.

13. Qualitative Research • Qualitative research is associated with a number of overlapping philosophies and methodologies such as naturalism, interpretivism, ethnography and phenomenology. It is not uncommon for qualitative researchers to draw on all of these and it may not be easy, for instance, to determine why a study has been described as ethnographic rather than phenomenological. • Hammersley & Atkinson (1983) define naturalism: • Naturalism proposes that the social world should be studied in its ‘natural’ state, undisturbed by the researcher. Hence, ‘natural’, not ‘artificial’ settings like experiments or formal interviews, should be the primary source of data. Furthermore, the research must be carried out in ways that are sensitive to the nature of the setting. • Interpretivism incorporates naturalism and emphasise the point that researchers can only understand the social world through their senses and a process of subjective interpretation. As a result, researchers never have direct access to the experiences, understandings and meanings of their subjects whose views are represented by research documents but only as interpreted by the researchers.

14. Qualitative approach can enable researchers to explore issues in depth. An experiment may highlight the clinical effectiveness of a medication, but it is less suited to exploring the experiences, opinions and feelings of patients who have to decide between having a kidney transplant or haemodialysis. • Unstructured data collection methods such as participant observation and semi-structured interviews are used in qualitative studies. • Qualitative researchers often draw on the grounded theory when undertaking sampling procedures, data collection and data analysis. This ensures that the theories that researchers develop are grounded in the data obtained 14

15. Pilot Work • It is good practice in quantitative or qualitative research to carry out a pilot study. This procedure serves several purposes such as: • Determining the length of time it takes to complete the entire instrument. • Identifying any parts of the instrument that are difficult for participants to understand or that have been misinterpreted by them • Determining needs for the training of the data collecting staff • Determining if the instrument yield data with sufficient variability • An example is the following questionnaire item. • “A patient is for a unit of blood. The blood was delivered at 1600hrs. What is the best time at which the transfusion should start?” Nurses provided various responses such as "as soon as possible", "1600hrs", "immediately", "1610 after warming the blood“. Based on these responses, options were introduced as follows. “A patient is for a unit of blood. The blood was delivered to the ward at 1600hrs.What is the best time by which the transfusion should start?” a. 1610hrs b. 1620hrs c. 1630hrs d. 1640hrs 15 15

16. Another example is the following: A patient is for blood transfusion. His blood group is A Rh D positive. While collecting the blood bag, the nurse noticed that the unit is A Rh D negative. What action should he/she take FIRST if the collected blood is compatible with the patient blood? • Inform the doctor b. Transfuse the unit after checking details c. Check details with another nurse- then transfuse the unit d. Refuse to transfuse the unit e. Initiate the transfusion, but observe and monitor the patient closely The problem with this item is that it was very difficult as only 3 out of 15 nurses provided the correct response. What’s to do in this case? I replaced “if the collected blood is compatible with the patient blood” with “PROVIDED THAT THE COLLECTED BLOOD IS COMPATIBLE WITH THE PATIENT BLOOD” • Research documents should state whether a pilot study was performed and give some idea of any problems encountered and what modifications were made. 16 16 16

17. A third example is the following: Specify which one of the following solutions/agents could be safely mixed with a blood transfusion. a. 5% Dextrose b. Lactated Ringer c. Normal saline 0.45% d. Normal saline 0.9% e. Drugs The problem with this question is that it was too easy with a difficulty index of .93 as 14 out of 15 nurses provided the correct response. Therefore, the item was modified as follows: Specify which ones of the following solutions/agents could be safely mixed with transfusion of packed red blood cells. a. 5% Dextrose water b. Normal saline 0.9% c. Morphine 10 mg/mL in normal saline d. Lactated Ringer e. Morphine 1.0 mg/mL in normal saline • Research documents should state whether a pilot study was performed and give some idea of any problems encountered and what modifications were made. 17 17 17

18. Exercise 7 • Refer to the three articles. Did the researchers involved in the three studies conduct any form of pilot work? If so, what modifications were made for the main studies?

19. Sampling Procedures • Sampling procedures tend to be quite different in quantitative and qualitative research. • The aim of sampling in quantitative research is to select a sample that is representative of the study population, in qualitative research, however, the aim is to select participants who will allow a detailed and in-depth exploration of the issues under study. 19

20. Sampling in quantitative research • Probability sampling is the most effective way of selecting a sample that is representative of the target population. In probability sampling every, person in the population under study has an equal chance of being selected for the sample. There are three main types of probability sampling. - Simple random sampling uses a computer, a random number table or numbers drawn at random to select a sample from the sampling frame (a numbered list of the entire population under study). - Stratified random sampling divides the population into two or more strata, such as male/female, and a random sample is then drawn from each stratum. - Cluster sampling can be used in large-scale studies where compiling a sampling frame of all the diabetic patients, for example, would be difficult and expensive. It would be easier and cheaper to select a random sample of health clinics, thus giving access to “clusters” of diabetic patients, all of whom (or a random sample of whom) could be selected depending on the sample size required. 20 20

21. It is important, when using a sampling frame, to be sure that it represents the target population. For example, a register of nurses registered to practise is not a register of nurses in practice. As implied above, there are no existing sampling frames for some populations. Compiling a nationwide database of diabetic patients or nursing assistants is a major piece of work in itself. • Because of the expense, time and difficulty that may be involved in probability sampling, non-probability, but less representative, sampling is often employed. There are three main types of this sampling. • Convenience sampling adopts the most readily accessible people. Thus, a researcher performing a survey may decide to distribute questionnaires to the first 50 nurses seen in a hospital who meet the inclusion criteria. • Snowball sampling employs a a request to participate. This is sent to known contacts, who are asked to provide names of other people who will agree to participate. These, in turn, are asked to provide further names and so on. • Quota sampling involves the researchers identifying strata in the population and selecting a convenience sample that reflects the proportions of those strata. Thus, if a particular population is composed of 400 males and 600 females, researchers might aim to survey 40 males and 60 females. If the strata are composed of 500 nurses, 250 physiotherapists and 250 doctors, in which case 50 of those chosen would be nurses, 25 physiotherapists and 25 doctors.

22. Exercise 8 What types of sample were used in the studies described by Allen et al. and McBride? Do you think that the sizes of the samples were adequate? Do the authors say whether they took statistical advice on sample size?