Supporting States in Building a Child Outcomes Measurement System

1. Supporting States in Building a Child Outcomes Measurement System Prepared for the ECO Advisers Meeting February 2011

2. Two Frameworks • Child Outcomes Measurement System • Family Experiences and Outcomes Measurement System Early Childhood Outcomes Center

3. Child Outcomes Measurement System • The set of components a state needs to have in place to make full use of child outcomes data. • NOT just a data system or a data collection method. Early Childhood Outcomes Center

4. Purpose of the Framework • Provide a common language for ECO and other TA providers to use in discussing COMSs with states. • Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS. • Serve as the organizing structure for the state self assessment Early Childhood Outcomes Center

5. Framework and Self-Assessment • Framework • Set of components and quality indicators • Provides the structure for the self-assessment • Self-assessment • Scale that provides criteria for levels of implementation within each quality indicator • Rating assigned based on level of implementation within each indicator Early Childhood Outcomes Center

6. Process for Framework Development • Built off what we had learned from ECO work with states • Literature review • Repeated discussion and review internally and with 7 Partner States Early Childhood Outcomes Center

7. Framework Partner States Early Childhood Outcomes Center

8. COMSFramework Components Purpose Analysis Data Collection and Trans-mission Reporting Using Data Cross-system Coordination Evaluation Early Childhood Outcomes Center

9. Quality Indicator • Provides additional detail as to what constitutes quality implementation of the component. • 18 quality indicators across the 7 components Early Childhood Outcomes Center

10. Quality Indicators Components Elements Purpose Data Collection and Transmission Analysis Reporting Using Data Evaluation Cross-System Coordination a. State has… b. State has… c. State agency.. d. Representative.. e. State agency… f. State …… g. State provides… h. State has.. Early Childhood Outcomes Center

11. 3 Quality Indicators for Data Collection and Transmission • Data collection procedures are carried out efficiently and effectively. • Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. • State's method for entering, transmitting, and storing data is effective and efficient. Early Childhood Outcomes Center

12. The Self Assessment Early Childhood Outcomes Center

13. Purpose of the Self Assessment • Provide guidance to states on what constitutes a high quality child outcomes measurement system. • Assist states in setting priorities for improving their COMS • Provide information to assist states in advocating for resources for systems development Early Childhood Outcomes Center

14. Each QI has multiple elements. Evidence for the extent of implementation for each element is provided. Each element is rated as NY = Not Yet or Don’t know IP = In Process, or IF = Fully Implemented The QI is given a rating based on the ratings of the elements. Early Childhood Outcomes Center

15. Back-up for Each Element Live link from the element Describes the element Describes what “fully implemented looks like” Provides examples of what “In process” might look like Provides examples of how states are addressing the element Provides additional resources related to the element Early Childhood Outcomes Center

16. The Scale for the Quality Indicators Early Childhood Outcomes Center

17. Early Childhood Outcomes Center

18. Web-based Scoring? Information can be entered online Will be stored for state by ECO Score from the QI page will automatically transfer to the overall score page Online scoring will show multiple marking points (i.e., change over time) Early Childhood Outcomes Center

19. Technology Support • Online capability • Store the evidence (text) • Transfer QI rating to the score sheet and generate bars • Also, allows ECO to use the aggregate state data to track where states are overall in building COMS • Downloadable version with many of the same features (Except ECO access to data) Early Childhood Outcomes Center

20. Working with the Self Assessment Early Childhood Outcomes Center

21. Possible Process • State identifies the highest priority QI(s). • Stakeholders convened • Priority QIs reviewed and rated • Plan developed to address elements not yet fully implemented. • Progress reviewed with stakeholders at regular intervals. • State identifies next set of QI(s), etc. Early Childhood Outcomes Center

22. Our questions for you • General impressions? • Content: Comprehensive? Appropriate? • Format/layout: User friendly? Understandable? • Length? • Scoring: Reasonable guideline? • Investment of resources • Continue development? • Pick new partners? • Pursue the web version? Heavily? Lightly? • Pursue the downloadable Word version? • How important is it that ECO track where states are in building their systems? Early Childhood Outcomes Center

23. For more information • For updates to the framework and the self-assessment and resources to support the quality indicators: www.the-eco-center.org Early Childhood Outcomes Center