Identifying and Selecting Measures for Health Disparities Research

1. Identifying and Selecting Measures for Health Disparities Research Anita L. Stewart, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities EPI 222, Spring April 22, 2010

2. Inappropriate Measures can Result in: • Conceptual inadequacy • Measuring wrong concept • Poor data quality (e.g. missing data) • Poor variability • Poor reliability and validity • Inability to detect true associations • e.g., no measured change in outcome when change occurred

3. Measure, Scale, Index, Item • Measure - single- or multi-item scale or index (umbrella term) • Scale - aggregation of items from one concept, scored using accepted scaling method • Index - aggregation of 2 or more scales into a summary score • Item - a single question or statement including its response scale

4. Composition of an Item During the past month, how much of the time have you felt tired?1 Never 2 A little of the time 3 Some of the time 4 Most of the time 5 All of the time Itemstem Response scale

5. Measure, Instrument, Battery, Questionnaire, Survey, Tool, Inventory • Measure - single- or multi-item scale or index (umbrella term) • Instrument - a published, named measure or set of measures • Battery - collection of measures from diverse sources

6. Single-item Measures - Usually Ordinal • Advantages • Response choices interpretable • Disadvantages • Impossible to assess complex concept • Very limited variability, often skewed • Reliability usually low

7. Multi-Item Measures or Scales Multi-item scales are created by combining two or more items into an overall measure or scale score Sometimes called summated ratings scales

8. Advantages of Multi-item Measures (Over Single Items) • More scale values (improves score distribution) • Reduces # of scores to measure a concept • Improves reliability (reduces random error) • Reduces % with missing data (can estimate score if items are missing) • More likely to reflect concept (content validity)

9. Interpersonal Processes of Care: Multi-item Measures I. COMMUNICATION Hurried communication (5) Elicited concerns, responded (3) Explained results, medications (4) II. DECISION MAKING Patient-centered decision-making (4) III. INTERPERSONAL STYLE Compassionate, respectful (5) Discriminated (4) Disrespectful office staff (4)

10. Selecting Measures for Your Research • Goal: find a measure of your concept that has been developed using stringent measurement development methods • Your task: find measures and review them for all steps in measurement development process

11. Process of Selecting Good Measures for Your Studies Define concept (variable) Identify potential measures Review measures’ properties --conceptual adequacy --psychometric adequacy Pretest best 1-2 measures Select final measure

12. Process of Selecting Good Measures for Your Studies Define concept (variable) Identify potential measures Review measures’ for --conceptual adequacy --psychometric adequacy Pretest best 1-2 measures Select final measure

13. Concept/Construct • A variable that is relatively abstract as opposed to concrete • e.g. health status, stress, acculturation vs. height, body temp • An abstraction based on observations of certain behaviors or characteristics • Cannot be assessed directly

14. Measures of Concepts • Concepts are defined and operationalized in terms of observed indicators or measures • Measures are “proxies” for the latent variables we cannot directly observe

15. Define Concept For Your Study • Define concept from your perspective, taking into account your… • study questions • target population • For outcomes, describe: • how intervention or independent variables might affect it • specific types of changes you expect

16. Example of Concept: Interpersonal Processes of Care • Interpersonal processes between physicians and patients • Communication, decision making, respectfulness, empathy • Emphasis on processes relevant to vulnerable patients

17. Purpose of Measuring Concept • Describe how your concept fits into your research question • Outcome measure? • Determinant of health? • Identify need for intervention?

18. Example: Interpersonal Processes of Care as an Outcome • Interpersonal processes of care may influence patient outcomes • Proximal outcomes • Knowledge of condition and recommendations • Adherence to recommendations • Patient satisfaction • Ultimate outcomes • Health status

19. Specify Context for Measures • Study characteristics affecting choice of measures • Nature of target population • Practical constraints • Nature of population (patients) • Lower educational level? Limited literacy? • Healthy or ill?

20. Process of Selecting Good Measures for Your Studies Define concept (variable) Identify potential measures Review measures’ for --conceptual adequacy --psychometric adequacy Pretest best 1-2 measures Select final measure

21. Locate Potential Measures • Identify candidate measures for all concepts • Redundancy OK for now • DO NOT develop your own questions unless it is absolutely necessary

22. Locating Potential Measures • Multi-item measures with known psychometric properties • Most good measures have been published • Original development and testing information • Information on application of measure in subsequent publications

23. Locating Potential Measures • Electronic databases • Compendia (reviews of measures) • Organizations and research centers • Government agencies • National and state surveys • Universities and individual researchers • Wikipedia (new)

24. Why Not Just Search on PubMed? • Can search PubMed using your concept term and MeSH terms questionnaires, measures, etc. • Terminology is not always intuitive • Articles are indexed by most specific MeSH term available • Cumbersome!

25. Measurement and Methods Cores Resources CADC (see handout): • http://dgim.ucsf.edu/cadc/cores/measurement/index.html RCMAR coordinating center • http://www.rcmar.ucla.edu/wiki/index.php/Measurement_Workgroup

26. Locating Published Measures: Compendia • Specific measures of various concepts are compiled, reviewed, listed, or otherwise provided • Many books and some special journal issues review and critique various measures • Reviews of measures are extremely helpful • Try to find these if possible

27. Best Compendium • McDowell I and Newell C, Measuring Health: A Guide to Rating Scales and Questionnaires.Third Edition,Oxford University Press, 2006. • Reviews measures of physical disability, social health, psychological well-being, anxiety, depression, mental status, pain, and general health status and quality of life • Includes actual instruments where possible and selected items in some

28. Compendia by Web • National Cancer Institute website • Health behavior constructs: theory, measurement, and research • Reviews concepts and measures of constructs such as perceived control, social support, and perceived vulnerability http://dccps.cancer.gov/brp/constructs/index.html

29. CINAHL (Nursing and Allied Health Literature) • CINAHL - on UCSF library home page • To find several potential measures • Use “questionnaires, scales, self-report, or surveys” in Major Subject Heading fieldANDyour topic area in another field (e.g., Abstract) • To find articles using a specific measure • Use name of measure and Instrumentation field

30. Google Scholar • http://scholar.google.com/ • More practical for searching on vague terms • Follow up on PubMed

31. Consider Alternative Labels to Search • Search on your term – see what other terms come up • Housing instability • Homelessness • Clinical assertiveness • Interpersonal behavior

32. Compendia by Web • National Cancer Institute • Health behavior constructs: theory, measurement, and research • Reviews concepts and measures of constructs such as perceived control, social support, and perceived vulnerability http://dccps.cancer.gov/brp/constructs/index.html

33. Locating Measures: Organizations and Research Centers • Some organizations and centers specialize in measurement development • RAND • MacArthur Research Network on Socioeconomic Status and Health

34. RAND Health Program • Measures, scoring manuals, and publications for measures of: • Health-related quality of life • Quality of care, patient satisfaction • Mental health • Maternal, child, and adolescent health • Medical Outcomes Study (MOS) measures of functioning and well-being http://www.rand.org/health/surveys_tools.html

35. MacArthur Research Network on Socioeconomic Status and Health • Reviews measures in several domains: • Psychosocial • Social and physical environment • Socioeconomic status (SES) • SES across the lifecourse http://www.macses.ucsf.edu/Research/overview.htm

36. Anxiety Coping Depression Discrimination Hostility Optimism/pessimism Personal control Psychological stress Purpose in life Self-esteem Social support Vitality and vigor Examples of MacArthur Network Psychosocial Measures http://www.macses.ucsf.edu/Research/wgps.htm

37. Society of General Internal Medicine • Dataset compendium – has links to numerous datasets such as: • National Survey of Children’s Health (NSCH) • California Health Interview Survey (CHIS) • Coronary Artery Risk Development in Young Adults (CARDIA) • Each link has section for data collection forms (questionnaires) • Sometimes involves several layers http://www.sgim.org/index.cfm?pageId=864

38. Ottawa Health Decision Centre • Patient and physician decision aids • http://www.ohri.ca/centres/DecisionAids/default.asp • Patient measures • Decisional conflict, decisional regret, stage of decision making, decision self-efficacy, and others • http://decisionaid.ohri.ca/eval.html

39. Commonwealth Fund Surveys • Health insurance, Medicare • Health system performance • Health care quality, patient centered care, underserved populations • Child health, care of the elderly http://www.commonwealthfund.org/surveys/

40. Locating Measures: Government Agencies • Several federal and state government agencies provide measures • Information about measures (e.g. clearinghouse) • Actual measures

41. National Center for Health Statistics (NCHS) • Surveys and data collection systems • Can download • Any survey or portion of survey • Handouts: • Summary of surveys August 2008 http://www.cdc.gov/nchs

42. NCHS National Health Care Surveys: Surveys of Physicians • Family of provider-based surveys • Provide objective, reliable information about • organizations and providers • services rendered • patients they serve http://www.cdc.gov/nchs/surveys.htm

43. Centers for Disease Control and Prevention • Behavioral Risk Factor Surveillance System (BRFSS) • Surveys of health behaviors www.cdc.gov/brfss/questionnaires/index.htm

44. CDC: National School-Based Youth Risk Behavior Survey (YRBS) • Survey conducted every other year • random national sample of youth in grades 9-12 • Most states conduct survey • Measures of substance use, risky sexual behaviors, diet, physical activity, overweight http://www.cdc.gov/HealthyYouth/yrbs/index.htm

45. California Health Interview Survey (CHIS) • A behavioral risk surveillance survey modeled after the CDC BRFSS • Adults, adolescents, and children • Numerous languages http://www.chis.ucla.edu/questionnaires.html

46. U.S. Dept of Veterans Affairs • National Center for Post Traumatic Stress Disorder • http://www.ptsd.va.gov/ • Assessment/Instruments • Trauma Exposure Measures • PTSD Screens • Adult Self Report

47. Agency for Healthcare Research and Quality (AHRQ) • National Quality Measures Clearinghouse • Consumer Assessment of Health Plans Survey (CAHPS) http://www.ahrq.gov/qual/

48. National Cancer Institute • Special initiatives on measures • Health Information National Trends Survey (HINTS) • Measures of cancer knowledge, cancer communication, risk perception http://hints.cancer.gov/

49. Locating Measures: Universities and Individual Researchers • Individual investigators who specialize in measurement and some universities post measures • Some are large research studies on university websites

50. Locating Measures: Large Research Studies • Many large-scale, multi-center and longitudinal studies have developed and used measures on health-related topics • Increasingly, they are posting these on “study” websites