Using Information & Data In Planning and Measuring Progress

Using Information & Data In Planning and Measuring Progress

Using Information and Data in Planning and Measuring Progress Changing the Culture of the Workplace • Mission and Vision • Family-centered practice. • Community-based services. • Individualized services to children and families. • Strengthening the capacity of parents to provide for their children's needs. • Changing the day-to-day practice of caseworkers in the field. • Building an infrastructure that supports changed practice.

Reports as Tools

Using Information and Data in Planning and Measuring Progress What Makes Good Data? Improving Data Quality Requires: • Clear definitions of good practice and desired outcomes • Clearly communicating expectations to staff • Teaching staff to enter correct data and to enter data correctly • Sharing information/data analysis • Requiring staff to understand and use data (timeliness and accessibility) • Flexible systems and measures • Increasing data quantity • Building reliance and trust in data through consistent usage

Using Information and Data in Planning and Measuring Progress What Makes Good Data? Improving Data Quantity Requires: • Accurate data mapping • Knowledgeable users • Effective administrative support • Strong policy definitions • Information analysis • Information sharing • Timely and complete entry by staff

Using Information and Data in Planning and Measuring Progress Reports as Tools Good reports answer the questions “What is being done and how well?” Reports can be grouped by type: • Resource Focused • Basic Case Measures • Resource Measures • Policy Focused • Compliance/Process Measures • Client Focused • Service Response Measures • Outcome Measures

Using Information and Data in Planning and Measuring Progress Basic Case Measures Basic case measures provide information on the amount of service being provided, and to whom. Examples are: • Number served • Client demographics • Services provided • Units of service • Case status Basic Case Measures

Using Information and Data in Planning and Measuring Progress Basic Case Measures Basic case measures provide certain insights that help with resource management. For example: • Staff resources needed to cover the number of cases receiving services • Demographics can provide insights on over or under representation of various groups of clients in the caseload • Identifying service delivery needs Basic Case Measures

Using Information and Data in Planning and Measuring Progress Resources Measures Resource measures provide information on the acquisition and expenditure of resources as well as efficiency of how resources are used. Major resources tracked may include: • Financial • Placement Search for relatives • Staffing Caseload standards

Using Information and Data in Planning and Measuring Progress Resource Focused Reports Examples of Resource Focused Reports: • Placement within licensed capacity • Caseload standards • List of staff trained on policy and procedures

Using Information and Data in Planning and Measuring Progress Compliance Measures: Compliance measures provide feedback to managers on adherence to agency policies and procedures. Examples of compliance measures include: • Commencement of investigation • Completion of investigation Compliance Measures: • Help insure standards of good practice are followed and that laws and regulations are carried out. • Focus on process rather than outcomes. • Are expressed both in case counts and rates that meet compliance.

Using Information and Data in Planning and Measuring Progress Client Focused Reports Examples of Client Focused Outcome Measures • Repeat maltreatment within 6 months • Maltreatment of children in care • Children achieving permanency within 12 months (reunification), or 24 months (adoption/guardianship) • Children re-entering care within 12 months • Multiple placements (no more than 2 within 12 months)

Using Information and Data in Planning and Measuring Progress Service Response Measures Service response measures, or outputs, are the results of actions taken that contribute to achieving outcomes. There is empirical evidence and a high degree of practice wisdom that doing these things result in improved outcomes or results. • Reduction in residential placements • Stability in placement • Possible supplemental service response measures: • Placement of children in proximity to their home • Parent-child visitation (in reunification cases)

Using Information and Data in Planning and Measuring Progress Priority attention to outcomes: Outcome and service response measures should be emphasized and given priority attention by all staff. Outcomes represent the centerpiece of child welfare performance. Other performance measures are important, but are subordinate to outcomes.

Using Information and Data in Planning and Measuring Progress • Data/Report Formats • The best reports are simple and easy to interpret. They may present • data in a number of ways for example they may present data as a: • Table of Numbers • A Graph • As Descriptive Text • Or as a combination of these formats A chart or graph should indicate a positive outcome, with very few exceptions, i.e.; child deaths. The indicator may need to be reversed to accomplish this. The number of children NOT re-entering care or the number of children NOT abused in out-of-home care.

Using Information and Data in Planning and Measuring Progress • Example of a Report in Table Format

Using Information and Data in Planning and Measuring Progress Example of a Report in Graphic Format With a Reverse Indicator

Using Information and Data in Planning and Measuring Progress Example of a Report in Text Format The incidence of abuse and neglect in out-of-home care has risen slightly over the past several months. This rise has has been less than one percent for all agency reporting districts with the exception of district number 2. The factors associated with this rise in district number 2 is being investigated. As soon as additional information is available it will be distributed.

Using Information and Data in Planning and Measuring Progress Effective Reports The most effective reports are ones that provide standards or expected levels of performance. • Draw standard lines on graphs - When data are graphed, a goal line on the chart gives the user the ability to instantly interpret their performance in relation to the standard. • Show the extent over or under - When data are presented in tables, an additional column can be provided that shows how far current performance is above or below standard.

Using Information and Data in Planning and Measuring Progress Specialized Reports • Detail Reports • Exception Reports • Distribution Reports • Count Down Reports • Trend Reports • Incident Counts

Using Information and Data in Planning and Measuring Progress A Trend Report Might Indicate Client data on re-entry might show: • They are generally teenagers, • They are in non-abuse/neglect cases, and • They come back into care within 3 months. While the data doesn’t indicate what to do, staff may want to look at: • How are they assessing readiness for reunification, • Possibly developing supervisory review of these cases, and • Intensifying aftercare efforts in the first 3 months for these kinds of cases.

Using Information and Data in Planning and Measuring Progress Quality Decisions Require Quality Data “The Decision Pyramid” Decisions Policy and Practice Knowledge & Analysis Data and Information

Using Information and Data in Planning and Measuring Progress Poor Decisions Are Often Based on Little or No Data “The Decision Pyramid” Decisions Policy and Practice Knowledge & Analysis Data Some people use the “decision pyramid” in reverse! Don’t confuse me with the facts. I know what to do!

Using Information and Data in Planning and Measuring Progress Continuous Quality Improvement

Using Information and Data in Planning and Measuring Progress Quality Assurance Reports • Exception reports: Identify missing data • Detail reports: What’s behind the numbers by worker/case/client • Cohort reports: Select specific groups of clients/cases • Clients served after a major change in policy • Cases closed following a change in procedure • Staff hired under new guidelines

Using Information and Data in Planning and Measuring Progress Quantity and Quality of Agency Data Factors that impact data quality and quantity: • Policy is well defined and communicated • Practice always follows written policy • The content of the case record is clearly defined • Someone assumes ultimate responsibility for the case record • Case record is reviewed regularly to insure accuracy • “If it isn’t in the “automated” record it didn’t happen!” Automation increases accountability for everyone!

Using Information and Data in Planning and Measuring Progress Using Outcome Indicators Focusing on the outcome indicators provides you with specific information on how these measures are computed. This will: • Enable you to be an informed consumer of the information; • Help you understand outcomes data in a way that you can improve the performance of your unit; and • Prepare you to advocate for good reports and access to data in your jurisdiction if you don't already have it.

Using Information and Data in Planning and Measuring Progress What Makes Good Data? Improving Data Quality Requires: • Clear definitions of good practice and desired outcomes • Clearly communicating expectations to staff • Teaching staff to enter correct data and to enter data correctly • Sharing information/data analysis • Requiring staff to understand and use data • Flexible systems and measures • Increasing data quantity • Building reliance and trust in data through consistent usage

Using Information and Data in Planning and Measuring Progress What Makes Good Data? Improving Data Quantity Requires: • Accurate data mapping • Knowledgeable users • Effective administrative support • Strong policy definitions • Information analysis • Information sharing • Timely and complete entry by staff

Using Information and Data in Planning and Measuring Progress Analyzing Data to Monitor Improvement Focusing on the TARGET!

Using Information and Data in Planning and Measuring Progress Analyzing Data to Monitor Progress • The goals that are set represent the priorities of your agency (outcomes versus just compliance), • Progress toward goals is measured in a way that provides accurate, valid, and timely feedback on outcome attainment (measures are well constructed, and good report systems are developed) • Decisions made are based on how to best achieve the desired outcome. Spreadsheet link!

Using Information and Data in Planning and Measuring Progress Putting It All Together

Using Information and Data in Planning and Measuring Progress Source: Iowa Department of Human Services, Data Informs Us Training

Using Information and Data in Planning and Measuring Progress Wrap-up

Using Information and Data in Planning and Measuring Progress Wrap Up and Evaluation • Closing comments • Final questions? • Evaluation • NRC-CWDT contact information: • Phone: 202/662-4285 • e-mail: nrccwdt@cwla.org • Web site information www.nrccwdt.org • Tips, Tools and Trends • Conferences • Other NRCs and additional Resources

Using Information & Data In Planning and Measuring Progress