Understanding Measurement in Empirical Communication Research

1. CMNS 260: Empirical Communication Research Methods4-Measurement (Part 1 of 2 slideshows) • systematic observation • can be replicated • requires: • construct (concept) • operational measure/instrument/tool for making empirical observations Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8)

2. Today’s Lecture • Review core notions: concepts, operational measures, empirical research, language of variables, hypothesis testing, errors in explanation, etc. • Reliability & Validity: relationship, types • Levels of Measurement • Scales & Indices (if time– material for this in another slideshow)

3. Recall:The Research Process Babbie (1995: 101)

4. “Dimensions” of Research Neuman (2000: 37)

5. The Research Wheel Choose Topic Focus Research Question Inform Others The “Research Wheel” Interpret Data Design Study Steps in the research process Collect Data Analyze Data Source: Neuman (1995: 12)

6. Developing research topics

7. Concepts Symbol (image, words, practices…) definition must be shared to have social meaning concepts with more than one possible value or attribute sometimes called variables

8. Concept Clusters Examples: Peer Group Role Model Broadcast Media Ethnic Identity Cultural trauma Collective memory Political economy

9. Measurement systematic observation can be replicated (by someone else) Measures include: Concepts (constructs), theories measurement instrument/tools Must recognize concept in observations (measures) ??(# of library holdings as a measure of quality of university?) MacLeans Magazine survey results, 2000.

10. From Concept to Measure Neuman (2000: 162)

11. Variables • Must have more than one possible “value” or “attribute” • Types: • dependent variable (effect) • independent variable (cause) • intervening variable • control variable

12. Causal Relationships • proposed for testing (NOT like assumptions) • 5 characteristics of causal hypothesis • at least 2 variables • cause-effect relationship (cause must come before effect) • can be expressed as prediction • logically linked to research question+ a theory • falsifiable

13. Errors in Explanation

14. Propositions logical statement about (causal) relationship between two variables i.e. “Increased television watching leads to more shared family time and better communication between children & their parents”

15. Types of Hypotheses (note: plural form of Hypothesis) • Null hypothesis • predicts there is no relationship • Direct relationship (positive correlation) • more time spent studying leads to higher grades • Indirect relationship (negative correlation) • More time spent playing video games leads to lower grades

16. Hypothesis Testing

17. Possible outcomes in Testing Hypotheses (using empirical research) • support (confirm) hypothesis • reject (not support) hypothesis • partially confirm or fail to support • avoid use of PROVE

18. Causal diagrams Direct relationship (positive correlation) X Y X Y Indirect relationship (negative correlation)

19. Spurious Association example

20. Causal Diagram R= Racism against non-whites D= Discrimination against non-whites S=Intelligence Test Scores S R D

21. Good & Bad Research Questions

22. Abstract to Concrete Concept to Measure

23. Reliability & Validity Reliability dependability is the indicator consistent? same result every time? Validity measurement validity - how well the conceptual and operational definitions mesh with each other does measurement tool measure what we think ?

24. Types of Validity

25. Content Validity measure represents all the aspects of conceptual definition of construct. how adequately a measure covers behavior representative of the universe of behavior the test was designed to sample. Love

26. Face & Expert Panel Validity judgement by group or scientific community that indicator measures the construct (conceptual def.) Examples: Socio-economic status (education, income & ?) Digital Divide (differences in access to computers, internet, broadband?...) Construct ? Measure Scientific Community

27. Criterion Validity The validity of an indicator is verified by comparing it with another measure of the same construct in which a researcher has confidence. Predictive : ex. Comparison of Aptitude test & performance measures concurrent validity: ex. Comparison of new measure with established one

28. Construct Validity A type of measurement validity that uses multiple indicators– the construct is a combination of measures of the same variable convergent: positive correlation with related measures discriminate: negative correlation with measures of different variables

29. Other Dimensions of Validity Internal Validity no error of logic internal to research design External Validity results can be generalized Statistical validity correct statistical methodology chosen ? assumptions fully met

30. Types of Reliability stability over time representative across different subgroups of a population equivalence multiple indicators intercoder type of equivalence reliability

31. Improving Reliability clearly conceptualize constructs increase level of measurement use pretests, pilot studies use multiple indicators : A B Empirical Independent Variable Measure Dependent Variable Measure Association? a1 a2 a3 b1 b2 Specific Indicators Specific Indicators

32. Relationship between Measurement Reliability & Validity reliability necessary for validity but does not guarantee it “necessary but not sufficient” measure can be reliable but invalid (ex. not measuring what you think you are measuring) Source: Neuman (2000: 171)

33. Quantitative & Qualitative“Trustworthiness”

34. Creating Measures Measures must have response categories that are: mutually exclusive possible observations must only fit in one category exhaustive categories must cover all possibilities composite measures must also be: uni-dimensional

35. Levels of Measurement Levels of Measurement • Categories (or attributes) must be exhaustive & mutually exclusive • Relations between levels --can collapse from higher into lower, not vice versa

36. Nominal Measurement different categories (names, labels, images) not ranked attributes are mutually exclusive and exhaustive. Examples: What media do you use for finding out about news? Television Newspapers Radio Magazines Internet Other Babbie (1995: 137)

37. Ordinal Measurement different categories (mutually exclusive, exhaustive) rank-ordered attributes indicate relatively more or less of that variable. distance between the attributes of a variable is imprecise Example: “How important are newspapers as your news source?”

38. Interval Measurement different categories ranked in order can also tell amount of difference between categories Babbie (1995: 137)

39. Ratio Measurement different categories ranked in order amount of difference between categories also possible to state proportion (have a true zero) Example: “What was your income in dollars last year?”

40. Examples

41. Continuous & Discrete Variables Continuous variables: can have an infinite number of values interval and ratio levels of measurement Discrete variables: distinct categories nominal and ordinal levels of measurement

42. Composite Measures (continued in second slide series) • Composite measures are instruments that use several questions to measure a given variable (construct). • A composite measure can be either unidimensional or multidimensional. • Ex. Indices (plural form of index) and scales