Meredith Cathcart Anne Kuschner Larry Edelman Patty Salcedo

1. Great Expectations: Practice and Policy for Developing a Quality Outcomes Assessment SystemA Tale of Two States Meredith Cathcart Anne Kuschner Larry Edelman Patty Salcedo Measuring Child and Family Outcomes 2008

2. California’s Child Outcomes System: Quality Assessment to Quality Programs • Overview of California’s DRDP Assessment System • Policies and practices that support quality assessment systems • Demonstration of an online learning module designed for new assessors • Use of data to improve services and outcomes

3. The Context of Diversity in California Size Geographic Ethnic Linguistic Socio-economic

4. Desired Results Access Project • Overall Leadership and Coordination of Assessment Efforts • Instrument Research and Development • Training and Technical Assistance in Assessment • Data Collection and Analysis • Support for Use of Data DR Grantees SEEDS SEECAP CPIN Statewide ECSE Conferences & Leadership Training ECSE Support for Regular Child Development Training and Technical Assistance for ECSE LEA Partners for Research & Training in Assessment California Department of Education/Special Education Division • Monitoring and Enforcement • State and Federal Reporting • Inter- and Intra-agency Coordination • Overall Direction • Policy Development • Funding

5. California Department of Education Desired Results System Assessment Instruments Child Development Division (CDD) Special Education Division(SED) • Desired Results Developmental Profile - Revised (DRDP-R) for: • Infant/toddler • Preschool • School-age • Preschool Desired Results Developmental Profile – Revised for Children with IEPs (PS DRDP-R) • Desired Results Developmental Profile access (DRDP access)

6. Purposes of the DRDP Assessment System • Assure that preschool-age children with IEPs and their families benefit from California’s ECSE services and programs • Provide information about children’s development in relation to research and state standards so that programs, services, and supports are designed to maximize children’s potential • Establish a “common ruler” for describing children’s progress across the state

7. Goals of the DRDP Assessment System Improve results for children and families Provide families with information about child progress and ways to support their children’s learning Provide teachers with authentic assessment information to inform program planning Provide program administrators, staff, and stakeholders with data to improve overall program design Enable California’s preschool special education system to comply with federal reporting requirements

8. Principles Guiding the Development of the DRDP Instruments • Desired Results are for all children • Content of the instruments is based on child development research • Instruments utilize an authentic assessment model and are based on observation in typical environments • Multiple sources of data should be gathered over time • Assessments are universally designed • Instruments include adaptations for children with disabilities

9. Basics of California’s System • In addition to the SPP, California must meet IDEA requirements for statewide assessment: • Every 3,4,and 5 year old is assessed 2x per year • Assessment windows cover 6 month intervals • The SELPA is the unit of accountability for reporting results • IEP teams determine which DRDP instrument to use and which adaptations should be in place • Data is reported using a statewide web-based system, SEDRS • T/TA system supports fidelity at all levels

10. Instruments Systems Change Training Collaboration California Goals & Federal Reporting Technical Assistance Reporting Data Family Participation Making Meaning of Data

11. Instruments Systems Change Training Collaboration California Goals & Federal Reporting Technical Assistance Reporting Data Family Participation Making Meaning of Data …and so on…

12. The DRDP Instruments for Preschool Special Education

13. Navigating the DRDP Instruments Our first online module in learning to use the DRDP instruments

14. DRDP Adaptations • Augmentative or Alternative Communication System • Alternative Mode for Written Language • Visual Support • Assistive Equipment or Device • Functional Positioning • Sensory Support • Alternative Response Mode

16. Training, Technical Assistance, and Professional Development

17. Training Facilitates Quality Assessment Practices • Teachers understand the requirements and responsibilities of the DRDP Assessment System • Teachers feel confident and competent using the DRDP assessment tools • The quality of data collected increases with fidelity to the use of the instrument

18. What Worked for the First 2 Years of Implementation • Creating a phase-in of the system over several years • Creating a training cadre drawn from highly qualified, experienced statewide trainers • Built on collaboration between ECSE T/TA contractors, SELPAs, and CDE • Systematically mentoring less experienced trainers • Conducting a mandatory Train-the-Trainer institute • Piloting the training sessions • Training trainers on the use of technology • Insisting on minimum standards for training facilities

19. Current Training System Components • Hold two statewide trainings annually for new assessors • Support and build capacity for local training events • Provide online training modules for assessors and data entry personnel • Provide online guidance documents, teacher resources, and discussion forum and listservs • Disseminate updates at statewide and local conferences and events • Communicate with SELPA and link with CDE

20. A Brief Tour of www.draccess.org

21. Making Meaning of DRDP Data

22. What Data is Collected? • Every preschool child with an IEP is assessed on the ten DRDP Indicators twice per year • Current demographic information is gathered on each child at each assessment period • Raw data is submitted by assessors or SELPAs to SEDRS • Scale scores are generated by SEDRS and reported back to SELPAs, district administrators, and service providers

23. Primary Uses of Assessment Information • Serves multiple purposes: • accountability • monitoring of child progress • program planning • Provides immediate, useful feedback about an individual child’s progress and status • Assessment scores are easily understood by staff and families, e.g.: • plot a child’s developmental trajectory • display status relative to typical development

24. The DRDP Contributes to: • Formative assessment: • information is gathered on an ongoing basis • observation of child behavior is conducted in typical routines and activities • information is gathered from families, child care providers, and/or related service providers • Summative assessment • data relative to the three OSEP child outcomes is aggregated by CDE and sent to OSEP • data for each SELPA is reported to the public according to OSEP requirements • data for programs is be provided through the use of DRDP scale scores

25. The blue colored band represents the typical developmental range for the child’s age The black line marks the place represented by the child’s scale score in that particular indicator 25

26. This report depicts the change between two assessment periods. 26

27. Group Measures into Indicators

28. Facilitated Process: Reflective Data Analysis (pilot 2008-09) When data analysis is done within the context of the team it yields useful information to programs and adds to increased data quality. Build time to review and analyze data into team meetings. Build a classroom and program summary Ask questions about the completeness and accuracy of the data Ask questions about the implications of the results for program design, services, and staff development 28

29. Some Questions about Completeness and Accuracy of Data • If children have a developmental age less than 3 years they should NOT be assessed on the PS DRDP-R: • Are IEP teams using an appropriate decision-making process? • What training is needed for IEP teams? • Assessors should be able to rate 90% of Measures: • Do teachers understand the developmental constructs being measured by the DRDP? • What are the barriers to observing collecting data? • Most children with visual impairments should be assessed with visual supports: • What is the use of this adaptation in your program? • Is more support needed for teachers? 29

30. DRDP Results Curriculum Linkage IEP goals By Indicator By OSEP Outcome By Developmental Area By Learning Foundations/Standards Specialty Curriculum General Curriculum (commercial or locally developed) Learning Foundations /Standards Child Development Research Functional Academic Age level Standards-based

31. Current Activities

32. Current Activities • Continue activities to support quality professional development • Modify the DRDP access instrument to align with Learning Foundations/preschool standards and ongoing revisions to the PS DRDP-R • Increase support to local communities on professional development, data reporting, and data analysis • Expand activities to interpret results in useful ways at local and state levels • Create additional reports in the data collection and analysis software (SEDRS) to assist teachers, parents, and staff

33. Thank you!