From data to action : Developing & disseminating key indicators from the 2003 National Survey of Children’s Health

1. From data to action:Developing & disseminating key indicators from the 2003 National Survey of Children’s Health Kathleen S. O’Connor Christina D. Bethell Stephen J. Blumberg June 27, 2007 Inaugural Conference of the International Society for Child Health Indicators Chicago, IL - June 26 - 28, 2007 Centers for Disease Control and Prevention National Center for Health Statistics

2. Overview • (Selected) efforts to publish child health indicators • General (selected) measurement issues to consider • The 2003 National Survey of Children’s Health (NSCH) model for key indicator development & dissemination • Survey methods • Expert panel review • Indicator development & decision-making • Indicator dissemination • Data Resource Center (DRC) on Child and Adolescent Health (www.childhealthdata.org)

3. (Selected) efforts to publish child health & well-being indicators • Kids Count • ChildTrends Data Bank • Index of Child & Youth Well-Being • National Longitudinal Study of Adolescent Health • Youth Risk Behavior Surveillance System • American Community Survey • National Survey of America’s Families

4. Measurement issues • Increasing demand for child health and well-being data, especially at the sub-national level • Lack of data: states & local areas • Small cell sizes: limited utility for subgroup analyses depending on research question • When comparing the “same” child health indicator from more than one survey: must compare question wording, construct definition, data collection date, length of referent period, et cetera

5. Measurement issues (2) • If examining trend data: • If estimates from two iterations of the same survey vary, has the population itself changed over time? Or could it be due to (for example) minor changes in question wording? What if this has policy implications? • Should these data be ‘standardized’ before running trend analyses? • Are two data points a ‘trend’ or a ‘comparison’? • Do data users want this much detail - will a non-researcher care?

6. Measurement issues (3)*(*adapted in part from McGlynn, Health Affairs 1997 16: 7-21). • Various stakeholders may perceive a key health concept differently (examples: definitions of disability, health care quality) • Challenges: • Identify & balance differing perspectives among stakeholders • Define accountability & framework • Establish judgment criteria • Identify reporting requirements • Develop information systems to support reporting requirements

7. NSCH, 2003 • Goal: produce uniform, comparable national & state estimates on health & well-being for children, families & communities • Sponsor: Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB) • Conducted by: CDC’s National Center for Health Statistics (NCHS), State and Local Area Integrated Telephone Survey (SLAITS) • Sampling frame of the National Immunization Survey (NIS) • Gain in efficiency, reduced survey cost

8. NSCH, 2003 • Survey yields over 100 indicators of child health & well-being in the following areas:: • Child’s HEALTH STATUS - physical, emotional, dental, CSHCN Screener • Child’s HEALTH CARE – including medical home • Child’s SCHOOL & ACTIVITIES • Child’s FAMILY – including maternal health status • Child & Family’s NEIGHBORHOOD

10. NSCH, 2003 • Currently the largest representative national survey of health, well-being, & healthcare of youth < 18 years old at the national & state levels • Data collection repeated 2007 • Parent/guardian respondent, household (HH) survey • N = 102,353 (~2,000 per state & DC) • Microdata publicly released in 2005

11. Sample design • Broad cross-sectional multistage sampling design • Random-digit-dial (RDD) telephone survey of HH (screened ~1.9 million telephone lines to identify eligible HH) • One child in HH randomly selected • Sampling weights adjusted for non-response & non-coverage • Weights permit national and state-level estimation

12. Data utility: NSCH • Multiple stakeholders • Title V MCHB performance objectives or ‘core outcomes’ for the general pediatric population; program evaluation • Main data source for a number of indicators re: the Title V block grant application & state MCH funding ( • Researchers • Policymakers • Journalists • Interested in new data & trends • Asks ‘How does this compare with __’ to gain an angle & perspective • Parents & advocates for children • Ask ‘why’?, want to know practical uses of data • Could support parental decision-making • Provide consumers with basic information to understand & use health care (access, utilization, etc.)

13. Use of expert panels: key indicator development, 2003 NSCH • Two expert panels convened by HRSA: • National Expert Panel (NEP) • Technical Expert Panel (TEP) • Third panel: Data Resource Center Advisory group (to be discussed) • NEP: identified key constructs that should be considered for inclusion • TEP: provided guidance on instrument & survey design • TEP = subset of NEP

14. Expert panels (2) • Provided guidance • Reviewed instrument drafts • Developed strategies & criteria to evaluate proposed methods & questions • Provided connections to relevant research; identified gaps in knowledge • Advanced progress of the survey design process • Represented broad-based, meaningful constituency involvement

15. Expert panels (3) • Range of specialties: practitioners, researchers, survey methodologists, parent advocates, pediatricians, Federal staff from two agencies • Successfully achieved wide representation of collective perspectives & experiences • Breadth & depth of competence that matched evaluative objectives (i.e., key indicators to be developed) • Highly collaborative & collegial process

16. Expert panels (4) • Process fulfilled Federal need to manage agency research, evaluation & dissemination efforts to improve pediatric health care for all children • Dimensions: timing: NEP mostly involved in early & very end stages; TEP highly involved throughout involvement: TEP offered numerous structured & unstructured opportunities for input (FTF, conf calls; NEP primarily FTF)

17. Expert panels (5) • TEP offered prioritized recommendations for sponsor decisions • Final decisions rested with HRSA/MCHB Dimensions

18. Indicator dissemination • Four chartbooks published by HRSA/MCHB with national & state-level indicators for youth & families • General health, well-being • Overweight • Rural health • Oral health • Special edition of Pediatrics solely dedicated to NSCH findings • Volume of manuscripts & posters indicate high level of knowledge production

19. “Build it and they will come…” • Developed and led by CAHMI -- Child and Adolescent Health Measurement Initiative based at the Oregon Health & Science University in Portland OR • A third group of experts (National Advisory Group) provides ongoing guidance & approves of development of standardized indicators (some overlap of personnel with TEP) • Sponsored by HRSA/MCHB

20. www.childhealthdata.org

21. Data Resource Center on Child & Adolescent Health (DRC) Interactivedata resource center • Hands-on, centralized, user-friendly access to data • National Survey of Children with Special Health Care Needs (NS-CSHCN) • National Survey of Children’s Health (NSCH) • Resources & information about data • Examples of how other state & family leaders use these findings, background about the national surveys, resources about health of children

22. Available data output • Multiple indicator state profile tables • Single indicator “all states” comparison tables • Data graphs and tables for every indicator • Comparing an indicator across any two geographic areas and • Comparing indicators across subgroups of children by age, race, insurance status, income, family structure, health status, etc.

23. DRC • Education • “Real time” technical assistance to understand, interpret & use data • Online workshops • Opportunities to partner with other stakeholders to discuss, interpret & act on findings

24. DRC use Educate policymakers • Key policy issues for your state • Programs needs for information • Child health needs Advocacy • Pressure points in program budgets, priorities • Effective methods to present a case • Use of data in Fact Sheets, testimony, the media, to strengthen family stories Grant writing • Use of data to strengthen proposal Kristin Grimm, Spitfire Strategies

26. DRC website www.childhealthdata.org umbrella site for national SLAITS survey data

27. Example of multiple indicator state profile

28. Example of single indicator “all state” comparison table (option to sort by rank)

29. Example of single indicator data table comparing two geographic areas

30. Example of single indicator graph comparing two geographic areas

31. Example of single indicator graph comparing two geographic areas and three subgroups of children (by type of health insurance)

35. Examples of available information

37. Putting data into action • How “Action for Children North Carolina” used data from the DRC • 1. Basis for an issue brief • Outcomes by Income: low-income children fare worse on many indicators • Strengthen relationships with media • Sent the findings out to North Carolina’s Hispanic news outlets and others

38. Basis for an issue brief • Data from the DRC allowed AFC to look at health, early care & education outcomes for children by income • First time AFC could do this for a large number of indicators • Good opportunity to provide recent comparable findings on health insurance & child care subsidies

39. Strengthen relationships with local media AFC is now the ‘go-to’ source for information about children in NC Great source of data by race (a frequent media request)

40. Other ways ACF used the data • Various presentations • Created a ‘Positive Indicators’ section in their biennial North Carolina Children’s Index report; variables included: • read to daily by a family member • attends religious services • parent feels neighborhood is safe

42. At the population level, this estimate translates into over 1 million children in Texas without health insurance of any type at the time of the survey

45. Click BLUE TEXT

46. Take home messages • Significant investments (time, effort, money, et cetera) have been expended to collect these data • Valuable source of key child health & well-being, family & community indicators • The 2003 NSCH model demonstrates a successful indicator development & dissemination process that may be applicable to your situation

47. Thanks! Kathleen S. O’Connor, MPH koconnor1@cdc.gov CDC, NCHS slaits@cdc.gov Hyattsville, MD Christina Bethell, PhD, MPH, MBA bethellc@ohsu.edu Director, the Child & Adolescent Health Measurement Initiative (CAHMI) Portland, OR Stephen Blumberg , Ph.D. sblumberg@cdc.gov CDC, NCHS slaits@cdc.gov Hyattsville, MD