National Core Indicators

1. National Core Indicators Overview for the State of Maine Sarah Taub & Giusi Chiri Human Services Research Institute January 30, 2003

2. Purpose and goals • What is NCI? • How will participation in NCI impact the Quality of Life Consumer Survey? • How will other NCI activities benefit the State of Maine? • What do the results look like? • How are states using the NCI data?

3. Project Beginnings • NASDDDS and HSRI collaboration • Launched in 1997 • Seven field test states (plus steering committee) • ~60 candidate performance indicators • Development of data collection instruments

4. What has NCI Accomplished? • Nationally recognized set of performance and outcome indicators for developmental disabilities service systems • Reliable data collection methods & tools • Baseline and trend data at the state & national level • Benchmarks of performance

5. NCI Structure • Currently 22 states plus Orange County in Phase V (FY2003) • HSRI provides technical assistance under subcontract to NASDDDS • Subcommittees address specific issues • Meet with full steering committee annually • Funded primarily through state participation fees

6. WA WA VT VT ME MT MA MA SD WY WY RI RI PA CT CT PA IA IA NE NE IN IN UT IL IL WV WV DE DE KY KY NC NC AZ AZ OK OK SC Orange Orange County County AL HI HI Participating States Phase V

7. What are the Core Indicators? • Consumer Outcomes: • Employment (earnings, hours worked) • Community Inclusion • Choice and Decision-making (personal choices, support-related choices) • Self-determination (new) • Relationships • Satisfaction (with home and job)

8. What are the Core Indicators? • System Performance • Service Coordination • Family and Individual Participation (on boards of directors) • Utilization (types of services provided) • Financial Level of Effort • Cultural Competence (access to supports) • Access (relative to population)

9. What are the Core Indicators? • Health, Welfare, and Rights • Safety (mortality, injuries, crimes) • Health (routine exams) • Medications • Wellness (new) • Restraints • Respect/Rights

10. What are the Core Indicators? • Staff Stability and Competence • Staff Stability (turnover and vacancy rates, length of employment) • Staff Competence (new)

11. What are the Core Indicators? • Family Indicators • Information and Planning • Choice & Control • Access & Support Delivery • Community Connections • Family Involvement • Satisfaction • Family Outcomes

12. What are the data sources? • Consumer Survey • Family Survey • Adult Family Survey (at home, 18+) • Family Guardian Survey (out-of-home) • Children Family Survey (at home, <18) • Provider Survey • Staff Stability • Board Representation • System Data • Incidents • Mortality

13. What it means to be from…a new NCI state • Transition to new survey tool • Similar questions and structure • Some methodological differences but still comparable • Technical support re: training, administration, analysis • Risk adjustment • Adult Family Survey • 53% response rate

14. What it means to be from…a new NCI state • Provider and system data • Common definitions • MIS improvements • General • Participation on subcommittees • Collaboration with other states, particularly New England region

15. National Core Indicators Selected Results

16. Provider Survey • Staff stability • Rising trend in staff turnover over past three years • Turnover is higher in residential settings vs. day settings • In FY2002, turnover rates ranged from 33% (Hawaii) to 54% (Indiana)

17. Staff Turnover Rates FY1999 - FY2001

18. Provider Survey: Staff Stability

19. Board Representation (FY2001)

20. Adult Family Survey • Surveys of families with an adult family member living at home • 5567 total surveys (37% response rate) • Average age of respondent = 57 • 89% of respondents were parents • 96% were primary caregivers

21. Adult Family Survey

22. Adult Family Survey

23. Adult Family Survey:Choice & Control

26. Consumer Survey • 7917 surveys completed • 67% of people interviewed were able to respond to Section I

27. Level of MR

28. Nursing Facility 0.9 Other 3.7 Apartment Program 4.3 Specialized Facility 8.5 Foster Care or Host Home 9.9 Independent Home or Apt 18.6 Group Home 24.7 Parent or Relative Home 29.1 Place of Residence 0.0 20.0 40.0

29. Scale Development • SCALES are composite measures that sum together a set of question responses to determine the level of some theoretical construct. EXAMPLE: • The “Community Inclusion” scale was created by adding up the answers to questions such as “Do you go shopping?”, “Do you always eat at home, or do you sometimes go out to eat?” and so forth.

30. Internal Consistency • A scale can be considered a reliable measure if its internal consistency is  .70. • The statistic that assesses the scale’s reliability is called Cronbach’s alpha.

31. Comparisons Among States • The ANALYSIS of VARIANCE is a collection of techniques used to test for differences among more than two groups. • Post hoc (multiple comparison) tests provide information about which groups are different from each other. • Alternatively one can compare each state against the national average.

32. Consumer Survey Analysis • Four “scales” were created to combine sets of related items • All scales had alpha > .70 • Service Coordination = .80 • Community Inclusion = .89 • Support-Related Choices = .92 • Personal Choices = .95

33. Consumer Survey Analysis • Community Inclusion Scale • Goes shopping • Goes on errands or appointments • Plays sports or exercises • Goes out to eat • Attends religious services • Belongs to clubs orcommunity organizations • Goes out for entertainment

34. N.E. Results vs. National Average Community Inclusion Scale Above average No difference Below average VT (.84) RI (.77) CT (.79)

35. Consumer Survey Analysis • Support Related Choices Scale • Chose job or day activity • Chooses support staff at home • Chooses support staff at job/day activity • Chose service coordinator • Chose residence

36. N.E. Results vs. National Average Support-Related Choices Scale Above average No difference Below average VT (.84) RI (.77) CT (.79)

37. Consumer Survey Analysis • Personal Choices Scale • Chose roommate • Chooses daily schedule • Chooses what to do in free time • Chooses what to buy with spending money

38. N.E. Results vs. National Average Personal Choices Scale Above average No difference Below average VT (.84) RI (.77) CT (.79)

39. Areas of Strength • 92% of all respondents report that they have enough privacy • over 90% of respondents report that support staff treat them with respect • 94% satisfied with home • 96% satisfied with work/day program • Participation in community activities is generally high, ranging from 69% to 96%

40. Areas for Improvement • 77% of all respondents report that service coordinators get them what they need, compared with 90% in FY2001 • 48% of respondents reported “sometimes” or “always” feeling lonely • only 52% of women had a GYN exam in the past year and 7% have never had one

41. What Can You Do with the Information? • Include on your web site • Prepare annual reports • Develop provider profiles • Use with sister agencies • Use in allocation decisions • Use to spot red flags

42. How Are NCI States Using Their Data? • Pennsylvania – Independent monitoring and quality improvement • South Carolina – Core component of external monitoring • Wyoming – Annual reports, CMS review • Massachusetts – Strategic planning • South Dakota – Provider profiles • North Carolina – Health indicators

43. For More Information • Final Reports for Phase IV (FY2002) are available on HSRI’s website:www.hsri.org • Contact us with questions: • Staub@hsri.org • Gchiri@hsri.org