110 likes | 464 Views
Construct Validity And its Threats. “Boy, a few more like that and I’ll be ready for Gamblers Anonymous”. Jill Hoxmeier H615: Advanced Research Design October 10, 2013. Twin Problems of Construct Validity.
E N D
Construct ValidityAnd its Threats “Boy, a few more like that and I’ll be ready for Gamblers Anonymous” Jill Hoxmeier H615: Advanced Research Design October 10, 2013
Twin Problems of Construct Validity Construct Validity: making inferences from the sampling particulars of a study to the higher-order constructs they represent • Understanding them • Assessing them “Thinking without the positing of categories and concepts in general would be as impossible as breathing in a vacuum” (Einstein, 1949)
Meehl & CronbachConstruct Validity in Psychological Tests • Three steps to evaluate construct validity: • Articulation of a set of theoretical constructs and their interrelations • Developing ways to measure the hypothetical constructs proposed by a theory • Empirically testing the hypothesized relations • Evolution of “Attitude” • From a loose variable comprised of beliefs, norms, intentions… • … to Fishbein-ian perspective that attitude = person’s evaluation of behavioral consequence x strength of belief that behavior will lead to those consequences Cronbach & Meehl, 1955
Why do we need valid constructs? • Research cannot be done without constructs • Constructs are the central means we have for connecting the operations used in an experiment to theory • Construct labels carry social, political, and economic implications • The creation and defense of basic constructs is a fundamental task of all science
Construct Validity • Fostered by: • Starting with a clear explication of the person, setting, treatment, and outcome constructs of interest • Carefully selecting instances that match those constructs • Assessing the match between instances and constructs for “slippage” • Revising construct descriptions accordingly
Understanding Psychological Domains “…there are often no natural units of measurement…” (Shonkoff & Philllips, 2000, p.83-83) • Pattern-matching logic: does the given instance sufficiently match the prototypical features to warrant using the category label?
How do you measure your domain? • Think about your own research interest… • What are the domains or constructs that are measured? • Have you discovered from the literature any challenges in measuring those domains? • What are ways investigators in your field have revised constructs to overcome those challenges?
Program of Research • A theory of constructs must emphasize • 1) operationalizing each construct several ways within and across studies • 2) probing the pattern match between the multivariate characteristics of instances and characteristics of the target construct • 3) acknowledge legitimate debate about the quality of the match given the socially constructed nature of both operations and constructs
Threats to Construct Validity • Inadequate explication of constructs • Construct confounding • Mono-operation bias • Mono-method bias • Confounding constructs with levels of constructs • Treatment sensitive factorial structure • Reactive self-report changes • Reactivity to the experimental situation • Experimenter expectancies • Novelty and disruptive effects • Compensatory rivalry • Resentful demoralization • Treatment diffusion What (potential) threats are most relevant to your research? And how do you plan to guard against such threats?
Questions for You! • What role does qualitative research play in contributing to construct validity? • How do we balance the theory construct testing of quantitative research and the “naturalistic generalizations” that may come from readers of “thick descriptions” that qualitative researchers provide?