Strengths and Limitations of Experiments

1. Strengths and Limitations of Experiments Controlled experiments are often considered the ideal of science. In this section, we explore the reasons why experimental methods are held in high esteem and examine some of their limitations.

2. Experimental Strengths True experiments are the most powerful method available for testing hypotheses of cause-and-effect relationships between variables. Studies using an experimental design are in general thought to yield the highest-quality evidence regarding the effects of specific interventions and nursing actions.

3. Because of their special controlling properties, experiments offer greater corroboration than any other research approach that, if the independent variable (e.g., diet, drug dosage, or teaching approach) is manipulated in a specified way, then certain consequences in the dependent variable (e.g., weight loss, recovery of health, or learning) may be expected to ensue. This “if... then” type of relationship is important because of its implications for prediction and control. The great strength of experiments, then, lies in the confidence with which causal relationships can be inferred.

4. Lazarsfeld (1955), whose thinking reflects the ideas of John Stuart Mill, identified three criteria for causality The first criterion is temporal: a cause must precede an effect in time. If we were testing the hypothesis that saccharin causes bladder cancer, it would be necessary to demonstrate that subjects had not developed cancer before exposure to saccharin. In experiments, researchers manipulate the independent variable and then measure subsequent outcomes, and so the sequence is under their control.

5. The second requirement is that there be an empirical relationship between the presumed cause and the presumed effect. In the saccharin and cancer example, we would have to demonstrate an association between saccharin consumption and the presence of a carcinoma, that is, that a higher percentage of saccharin users than nonusers developed cancer. In an experiment, this empirical relationship between the independent and dependent variables is put to a direct test.

6. The final criterion for establishing a causal relationship is that the relationship cannot be explained as being caused by a third variable. Suppose, for instance, that people who elect to use saccharin tend also to drink more coffee than nonusers of saccharin. There would then be a possibility that any relationship between saccharin use and bladder cancer reflects an underlying causal relationship between a substance in coffee and bladder cancer.

7. It is particularly because of this third criterion that the experimental approach is so strong. Through the controls imposed by manipulation, comparison, and randomization, alternative explanations to a causal interpretation can often be ruled out or discredited.

8. Experimental Limitations Despite the benefits of experimental research, this type of design also has limitations. First of all, there are often constraints that make an experimental approach impractical or impossible. These constraints are discussed later in this chapter. Experiments are sometimes criticized for their artificiality. Part of the difficulty lies in the requirements for randomization and then equal treatment within groups. In ordinary life, the way we interact with people is not random.

9. For example, certain aspects of a patient (e.g., age, physical appearance, or severity of illness) may cause us to modify our behavior and our care. The differences may be subtle, but they undoubtedly are not random.  Another aspect of experiments that is sometimes considered artificial is the focus on only a handful of variables while holding all else constant. This requirement has been criticized as being reductionist and as artificially constraining human experience.

10. Experiments that are undertaken without a guiding theoretical framework are sometimes criticized for being able to establish a causal connection between an independent and dependent variable without providing any causal explanation for why the intervention resulted in the observed outcomes.

11. A problem with experiments conducted in clinical settings is that it is often clinical staff, rather than researchers, who administer an intervention, and therefore it can sometimes be difficult to determine if subjects in the experimental group actually received the treatment, and if those in the control group did not. It may be especially difficult to maintain the integrity of the intervention and control conditions if the study period extends over time.

12. Moreover, clinical studies are usually conducted in environments over which researchers have little control—and control is a critical factor in experimental research. McGuire and her colleagues (2000) describe some issues relating to the challenges of testing interventions in clinical settings. Sometimes a problem emerges if subjects themselves have discretion about participation in the treatment.

13. Suppose, for example, that we randomly assigned patients with HIV infection to a special support group intervention or to a control group. Experimental subjects who elect not to participate in the support groups, or who participate infrequently, actually are in a “condition” that looks more like the control condition than the experimental one. The treatment is diluted through nonparticipation, and it may become difficult to detect any effects of the treatment, no matter how effective it might otherwise have been. Such issues need to be taken into consideration in designing the study.

14. Another potential problem is the Hawthorne effect, which is a placebo effect. The term is derived from a series of experiments conducted at the Hawthorne plant of the Western Electric Corporation in which various environmental conditions, such as light and working hours, were varied to determine their effects on worker productivity. Regardless of what change was introduced, that is, whether the light was made better or worse, productivity increased.

15. Knowledge of being included in the study appears to have affected people’s behavior, thereby obscuring the effect of the variable of interest. In a hospital situation, researchers might have to contend with a double Hawthorne effect. For example, if an experiment to investigate the effect of a new postoperative patient routine were conducted, nurses and hospital staff, as well as patients, might be aware of their participation in a study, and both groups might alter their actions accordingly.

16. It is precisely for this reason that double-blind experiments, in which neither subjects nor those who administer the treatment know who is in the experimental or control group, are so powerful. Unfortunately, the double blind approach is not feasible for most nursing studies because nursing interventions are more difficult to disguise than drugs.

17. Example of a double-blind experiment McCormick, Buchman, and Maki (2000) used a double-blind approach (with a before-after design) to test the effectiveness of two alternative hand-care treatments (an oil-containing lotion and a novel barrier skin cream) for health care workers with severe hand irritation. Subjects in both groups experienced marked improvements. In sum, despite the clear cut superiority of experiments in terms of their ability to test causal hypotheses, they are subject to a number of limitations, some of which may make them difficult to apply to real-world problems. Nevertheless, with the growing demand for evidence-based practice, true experimental designs are increasingly being used to test the effects of nursing practices and procedures.

18. Reference • nursing research book 7th edition .