New Tools in the Tool Box: What We Need in the Next Generation of Early Childhood Assessments

1. New Tools in the Tool Box: What We Need in the Next Generation of Early ChildhoodAssessments Kathy Hebbeler ECO at SRI International Presented at the OSEP EC Conference, December 2008 Early Childhood Outcomes Center

2. Today’s Topics • The new National Academy of Sciences Report: Early Childhood Assessment: Why, What, How • What should the next generation of assessment tools look like? Early Childhood Outcomes Center

3. Early Childhood Outcomes Center

4. Reason for the Committee • Controversy over Head Start National Reporting System (NRS) • Congressional mandate • “review and provide guidance on appropriate outcomes and assessments for young children.” • 2 key topics: • Identification of key outcomes for children 0-5 • Quality and purpose of different state-of-the art techniques Early Childhood Outcomes Center

5. Organization of the Report • Overview of purposes • What to assess: Outcomes • And Quality of Environment • How to assess: • Technical discussion of validity • Issues related to minority, ELL, children with disabilities • Thinking systematically Early Childhood Outcomes Center

6. 2 Key Principles • The purpose of the assessment should guide assessment decisions • Assessment should be conducted within a coherent system of medical, educational, and family support services, that promote optimal development for all children Early Childhood Outcomes Center

7. Purposes • Determining an individual child’s level of functioning • Individual-focused screening • Community-focused screening • Diagnostic testing • Establishing readiness Early Childhood Outcomes Center

8. Purposes (Cont’d) • Guiding intervention and instruction • Evaluating the performance of a program or society • Program effectiveness • Program impact • Social benchmarking • Advancing knowledge of child development Early Childhood Outcomes Center

9. Guidelines from Multiple Groups • Assessment should benefit children • Assessments should meet professional, legal, ethnical, standards • Assessments should be age-appropriate or developmentally/individually appropriate • Parents/family should be involved in the assessment whenever possible • Assessments should be linguistically and culturally appropriate/responsive Early Childhood Outcomes Center

10. Guidelines from Multiple Groups (Cont’d) • Assessments should assess developmentally/educationally significant content • Assessment information should be gathered from familiar contexts (NEGP), realistic settings and situations (NAEYC), or be “authentic” (DEC). • Information should be gathered from multiple sources. • Screening should be linked to follow-up assessment. Early Childhood Outcomes Center

11. Recommendations Related to Purpose • Make assessment purpose explicit and public • Assessment strategy must match purpose Early Childhood Outcomes Center

12. [Which] Outcomes • Why this domain? • Evidence of consensus around value • Evidence of continuity within domain over development or links to other domains • Evidence that it is a target for intervention and affected by the environment Early Childhood Outcomes Center

13. Outcome: Domains (NEGP) • Physical well-being and motor development • Socioemotional development • Approaches to learning • Language (and emerging literacy) • Cognitive skills (including mathematics) Early Childhood Outcomes Center

14. Classification Challenges • Boundaries are artificial • Researchers (and test developers) use different classification schemes Early Childhood Outcomes Center

15. Recommendations related to Domains and Outcomes • Go beyond the standard domains of language and mathematics • Need support for better measures of socioemotional development • D-4 “For children with disabilities and specials needs, domains-based assessments may need to be replaced or supplemented with more functional approaches” Early Childhood Outcomes Center

16. Validity and Reliability • Weight of evidence • Not whether an assessment is valid and reliable • Use accumulation of evidence to judge whether the assessment is suitable for the task for which it is intended. • Evidence pertains to specific type of uses. Early Childhood Outcomes Center

17. Assessing All Children • Much more research needs to be conducted on the validity of tools with minority children, ELL, and children with disabilities. • Children with disabilities • Construct irrelevant skills as a threat to validity • Interrelatedness of domains • Functional outcomes • Universal design • Many tools used with children with disabilities have not been validated for the purposes for which they are being used. Early Childhood Outcomes Center

18. Instrument Selection Recommendations • Select assessment with acceptable levels of validity • Examine data produced with the assessment for validity for purposes for which the assessment is being used. • Test developers need to collect and make available evidence related to validity for minority, ELL, children with disabilities. • Be very cautious about reaching conclusions about a group not well represented in the validation sample. Early Childhood Outcomes Center

19. Situating Assessment in a System • EC System has • A goal for children • Strategies to achieve this goal • Infrastructure supporting the goal and the strategies • Selects assessment to be compatible with other elements of the system. Early Childhood Outcomes Center

20. Systems Recommendations • EC assessment system must be part of a larger system with strong infrastructure to support early care and education. • Standards • Assessment • Reporting • Professional development • Opportunity to learn • Inclusion • Resources • Monitoring and evaluation Early Childhood Outcomes Center

21. Systems Recommendations • Coherence • Horizontally coherent – curriculum, instruction, assessment aligned with early learning standards • Vertically coherent – shared understanding of goals at all levels; consensus about purposes of assessment • Developmentally coherent – taking into account what is known about young children’s skills and understanding develop Early Childhood Outcomes Center

22. Research Needed • New conceptual frameworks • Application of technical advances, e.g., IRT • Span birth to 6 or 7 • Social emotional measures • Capture children’s growth toward being able to meaningfully participate in variety of everyday setting • Use of technology Early Childhood Outcomes Center

23. Challenges with Current Assessments for All Children/Children with Disabilities • Domains-based • When should assessment look at only one domain? When should it look at the whole child? • How functional are current tools? • Varies • Need for tools to assess the three outcomes • How authentic? How real? • How much is lost with structured requests? And for which children? Early Childhood Outcomes Center

24. Challenges with Current Assessments for All Children/Children with Disabilities • Tool structure • E.g., Work Sampling • Need for specificity – sometimes.. • E.g., AEPS; High Scope COR • Do we want predictive validity? • Universal design principles • DRDP as the exception • Adaptations and accommodations Early Childhood Outcomes Center

25. Challenges with Current Assessments for All Children/Children with Disabilities • Diagnosis vs. intervention planning purpose • Must these be separate tools? • Are the things we want to know about children to prove eligibility so different from what we want to know to develop an intervention plan? • AEPS in JEI, December 2008 • Recognition and Response/RTI’s tools for young children • IGDI’s • How do these fit with curriculum-based assessment? Early Childhood Outcomes Center