1 / 31

Promoting Quality Data for Effective Program Improvement

This presentation by the Early Childhood Outcomes Center focuses on factors that improve and lessen the quality of data, and how to address these factors. It emphasizes the importance of ongoing quality assurance and provides training and support for data collection, analysis, and verification. The presentation also discusses the importance of team discussions, supervision, and reviewing completed forms for maintaining data quality.

kterence
Download Presentation

Promoting Quality Data for Effective Program Improvement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality Assurance: Looking for Quality Data I know it is in here somewhere Presented by The Early Childhood Outcomes Center Revised January 2013

  2. What factors work to improve the quality of your data? What factors work to lessen the quality of your data? How to address these factors? Activity Early Childhood Outcomes Center

  3. Take Home Message If you conclude the data are not (yet) valid, they cannot be used for program effectiveness, program improvement or anything else. What do you if the data are not as good as they should be? Answer: Continue to improve data collection through ongoing quality assurance Early Childhood Outcomes Center

  4. Many steps to ensuring quality data

  5. Many steps to ensuring quality data

  6. Promoting quality data Training and support before and during data collection Analysis of the data after data collection Data system and verification after data collection Early Childhood Outcomes Center

  7. Through training and communication related to: Assessment Understanding the COS process Age expectations Data entry Promoting Quality Data Early Childhood Outcomes Center

  8. Promoting Quality Data Through training materials, such as: Video team and child examples Written child examples “Quizzes” for ensuring learning Refresher trainings – Beware of Drift!! Early Childhood Outcomes Center

  9. Promoting Quality Data Through data systems and verification, such as: Data system error checks Good data entry procedures Early Childhood Outcomes Center

  10. Many steps to ensuring quality data

  11. Ongoing supervision Review of the process Is the process high quality? Are teams reaching the correct rating? Methods Observation Videos Early Childhood Outcomes Center

  12. Quality Review of COS Team Discussion Do all team members participate in the discussion? Is parent input considered in the rating? Give examples. Is the team documenting the rating discussion? Give examples. Does the team discuss multiple assessment sources? What are they? Early Childhood Outcomes Center

  13. Quality Review of COS Team Discussion Does the team describe the child’s functioning, rather than just test scores? Give examples. Does the discussion include the child’s full range of functioning, including skills and behaviors that are age appropriate, immediate foundational, and leading to immediate foundational? Give examples. Early Childhood Outcomes Center

  14. Quality review through process checks Provider surveys Self assessment of competence Knowledge checks Process descriptions (who participates?) Identification of barriers Kansas’ survey Alaska’s survey Early Childhood Outcomes Center

  15. Questions from Alaska’s Survey Early Childhood Outcomes Center

  16. Ongoing Supervision Feedback to teams is critical Refresher training Beware of: Auto pilot Drift Early Childhood Outcomes Center

  17. Quality Review of COS Team Discussion: Activity Observe team video Evaluate quality Early Childhood Outcomes Center

  18. Many steps to ensuring quality data

  19. Quality Review of Completed COS Forms Is the COSF complete? Is there adequate evidence for the basis for the rating? Does the evidence match the appropriate outcome area? Is the evidence based on functional behaviors? Early Childhood Outcomes Center

  20. Quality Review of Completed COS Forms Is there evidence that the child’s functioning across settings and situations considered? Are the ratings consistent with the evidence? Early Childhood Outcomes Center

  21. Quality Review of COS Forms: Activity Review completed COS Form with errors Early Childhood Outcomes Center

  22. Promoting quality data through data analysis Examine the data for inconsistencies If/when you find something strange, look for other data that might help explain it. Is the variation caused by something other than bad data? Early Childhood Outcomes Center

  23. The validity of your data is questionable if… The overall pattern in the data looks ‘strange’ Compared to what you expect Compared to other data Compared to similar states/regions/school districts Early Childhood Outcomes Center

  24. COS Ratings - Fall

  25. Outcome 3: Appropriate Action

  26. OSEP Categories

  27. Questions to ask Do the data make sense? Am I surprised? Do I believe the data? Believe some of the data? All of the data? If the data are reasonable (or when they become reasonable), what might they tell us? Early Childhood Outcomes Center

  28. Validity Validity refers to the use of the information Does evidence and theory support the interpretation of the data for the proposed use? Or Are you justified in reaching the inference you are reaching based on the data? Standards for Educational and Psychological Testing (1999) by American Educational Research Association, American Psychological Association, National Council on Measurement in Education Early Childhood Outcomes Center

  29. The validity of your data is questionable if: ? Early Childhood Outcomes Center

  30. ….not all providers are not knowledgeable about in the COS process …not all providers are careful with the COS process …the data look “strange” …etc. The validity of your data is questionable if: Early Childhood Outcomes Center

  31. Many steps to ensuring quality data

More Related