160 likes | 302 Views
COM 633: Content Analysis Measurement; Validity. Kimberly A. Neuendorf, Ph.D. Cleveland State University Fall 2010. Measurement. “Assignment of numerals to objects or events according to rules†(Stevens, 1951). Operationalization (Measurement).
E N D
COM 633: Content AnalysisMeasurement; Validity Kimberly A. Neuendorf, Ph.D. Cleveland State University Fall 2010
Measurement • “Assignment of numerals to objects or events according to rules” (Stevens, 1951)
Operationalization (Measurement) • Categories that are exhaustive and mutually exclusive • High and appropriate level of measurement • Ratio • Interval • Ordinal • Nominal
Evaluating Measures:Comparing Reliability, Accuracy (lack of Bias), and Precision • Targets…see p. 114 of book. • See also: • Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: Sage Publications.
Validity • Internal • Conceptual definition =matches= Operationalization • The standard of having a “good” measure—”Are we hitting the target we want?” • External • Generalizability • Sample • Ecological validity • Replicability
Validity • “Face” • Criterion • Relate to established standard or behavior • Concurrent • Predictive • e.g., Oliver & Kalyanaraman (2002) study of sexual and violent content in trailers and films. . . Criterion is the MPAA rating
Validity • Content Construct
Validity • Content
Validity • Content Construct = ???
Validity • Content High reliability. . .
Validity • Construct • Related to other measures (constructs) in ways consistent with theory • Convergent—relationships as expected • Discriminant—lack of relationships, as expected
Construct Validity • Smith volume (1992) on “thematic content analysis”—psychographic applications—has examples • Trust-mistrust coding system for TAT stories construct validated against: • Other thematic coding systems (e.g., Affiliation; Intimacy) • Demographics (e.g., Mothers’ feeding habits during infancy (more or less principled), moderated by gender & age) • Self-reported behaviors (e.g., Reported positive relationships with others; Divorce, family fights in last year; Drug use) • Personality measures (e.g., Impulsivity; Locus of Control)
Validity Assessment & CATA • Short et al. (2010) provides a model for validity and CATA, using tests of a new dictionary for “entrepreneurial orientation” • Content Validity • Deductive • Inductive • External Validity • Reliability • Dimensionality (a type of Construct Validity--Discriminant) • Predictive Validity (seems to really be Construct Validity—Convergent)