Research Control

Research Control

Research Control • Control that the investigator puts on the research situation and the subjects • Control extraneous variables to determine the true relationship between the IV and DV under investigation

Extraneous Variables • A variable that confounds the relationship between the IV and DV and that needs to be controlled in the research design or through statistical procedures

The most important question to ask is: • What variables may affect the dependent variable besides the independent variable?

Example • What may affect readmission to the acute care facility BESIDES adequacy of home care? • Look at the literature • Own clinical knowledge

Extraneous Variables • External- factors stemming from the research situation • Intrinsic- factors that are inherent to the subjects in the research study

Controlling the Research Situation (External Factors) • What can the researcher do to control the research situation itself? • Minimize situational contaminants • To make conditions under which the data are collected as similar as possible for every participant in the study

Controlling the Research Situation (External Factors) • Constancy of condition • Environmental context within which the study is to be conducted • Consider the setting type and emotional and behavioral aspects of the setting • Avoid contamination of treatment (control group and experimental group sharing with each other)

Controlling the Research Situation (External Factors) • Timing • Time of day or year • Strive for constancy of time for all the subjects • May or may not be critical to the variables

Controlling the Research Situation (External Factors) • Communications • Information about the purpose, how data collected, etc • Prepare in advance • Same message to all participants with little ad libbing • Given by the same person(s) whenever possible

Controlling the Research Situation (External Factors) • Research Protocols • Written down in great detail • Done by as few as possible trained research assistants • Adhered to specifications

Controlling Subject Characteristics (Intrinsic Factors) • Randomization • Most effective way to control • Secure comparable groups • Equalize the groups with respect to the extraneous variables

Controls all possible sources of extraneous variation without any conscious decision by the researcher about which variables need to be controlled • Expect that the experimental group and control group are comparable

Controlling Subject Characteristics (Intrinsic Factors) • Repeated Measures or Crossover Design • Same subjects serve as both control group and experimental group • Problem: carryover effect • Ordering of the treatment is important • Use randomized ordering

Controlling Subject Characteristics (Intrinsic Factors) • Homogeneity • Use of subjects who are homogeneous with respect to variables that are considered extraneous • Achieved through determining what possible variable influence the DV besides the IV • Determine eligibility and ineligibility criteria

Controlling Subject Characteristics (Intrinsic Factors) • Blocking • Controlling extraneous variable by including them in the design as a IV • Random assignment • Table 11.1 page 291 • Mostly used for experimental designs but can be used for non experimental

Matching • Statistical Control • Analysis of Covariance

Problems • Except for randomization, the researcher must know in advance what the extraneous variable are • Can practically only deal with 1-3 extraneous variables at a time (except analysis of covariance)

Validity • Does the design do the best possible job of providing trustworthy answers to the research question? • Statistical Conclusion Validity • Internal Validity • Construct Validity • External Validity

Validity Statistical Conclusion Validity Power Precision Answers the question: What is the strength of evidence that a relationship exists between the two variables.

Statistical Conclusion Validity • Power -- ability of the design to detect true relationships among the variables • Sample Size • Definition of the IV

Statistical Conclusion Validity • Precision • Accurate measuring tools • Controls over extraneous variables • Statistical methods

Validity • Internal Validity • Extent to which it is possible to make an inference that the IV is truly influencing the DV and that the relationship is not spurious • Answers the question If a relationship exists, what is the strength of the evidence that IV of interest rather than extraneous factors caused the outcome?

Threats to Internal Validity • The nature of the true experiment lends itself to a high degree of internal validity because of manipulation and randomization • All other design the researcher must contend with competing explanations for the obtained results (internal validity)

Threats to Internal Validity • History - occurrence of external events that take place concurrently with the IV that can affect the dependent variable of interest • In a true experiment can assume that it affects both the control group and the experimental group similarly

Threats to Internal Validity • Selection- biases resulting from preexisting differences between the group • Group differences on the DV are the result of the groups not being equivalent in the first place rather than the effect of the IV • One of the most common, problematic threats to internal validity

Threats to Internal Validity • Maturation • Processes occurring within the subjects during the course of the study as a result of the passage of time rather than as a result of treatment or independent variable.

Threats to Internal Validity • Testing effect- the effect of the pretest on the participant’s performance on a post test • The mere collecting of information can change the subject’s attitude, behavior, etc

Threats to Internal Validity • Instrumentation- changes in the researcher’s measuring instruments between an initial point of data collection and a subsequent point • Keep the pre and post test the same • Consider changes in the people collecting the data, subject’s attitude toward the instruments on post test

Threats to Internal Validity • Mortality- differential attrition from the groups being compared • Becomes a consideration in the initial design of the study • If the attrition is random (those dropping out and those staying are similar bias is low • More than 20% attrition is a concern

Validity • External Validity achieved when the results can be confidently generalized to situations outside the specific research setting • Answers the question: If the relationship is plausibly causal, what is the strength of the evidence that the relationship is generalized across people, settings, and time?

External Validity • Refers to the generalizability of the research findings to other settings or samples • Aim of research is discover enduring relationships • To what populations, environments, and conditions can the results of this study be applied?

External Validity • Sampling design must be adequate • “Strictly speaking the findings of a study can only be generalized to the population of subjects from which a study sample has been selected at random.”

Threats to External Validity • Environmental or research situations that affect the study’ representativeness • Hawthorne Effect • Novelty Effect • Interaction of History and Treatment Effect • Experimenter Effects • Measurement Effects

Hawthorne Effect • Subjects may behave in a certain manner simply because they know they are being studied • Thus you cannot generalize to a more natural setting

Novelty Effect • New treatment might cause excitement of skepticism and thus alter behavior

Interaction of history and treatment effect • Impact of the treatment AND some other event external to the study

Experimenter effect • Subject’s reaction to the researcher

Measurement Effect • Collection of data and attention from having all the data collected

Research Control