Continuous Progress Monitoring for Young Children Judith J. Carta and Charles R. Greenwood

1. Continuous Progress Monitoring for Young ChildrenJudith J. Carta and Charles R. Greenwood

2. Objectives today • Provide some background on use of Individual Growth and Development Indicators for Continuous Progress Monitoring • Show an example of one IGDI: Early Communication Indicator • Show how the data can be used for making intervention decisions

3. Developers of IGDIs in Early Childhood • University of Kansas: Charles Greenwood, Dale Walker, Jay Buzhardt, Kathleen Baggett, Judith Carta—IGDIs for children birth to 3 years • University of Minnesota: Scott McConnell—IGDIs for 3-5 year olds • University of Oregon & Dynamic Measurement Group: Ruth Kaminski & Roland Good-DIBELS—Dynamic Indicators of Basic Early Literacy Skills

4. Rationale: Why Progress Monitoring? - Charlie • The best investment we can make today educating our children is early intervention! • E.g., Heckman, others • Public and private preschool programs are scaling up and becoming part of the USA’s K-12 system • Federal and state policies are requiring that programs and services produce child outcomes • Early Head Start and Head Start • Office of Special Education Programs • Programs are required to use evidence-based practice • Practices for which research indicates that children are more likely to benefit compared to other practices

5. Progress Monitoring Measurement • Because PMM is an evidence-based practice • Use of PMM leads to • Greater teacher understanding of what each children can do and needs to learn • Earlier identification of children not making progress • Increased teacher planning and changes in children’s intervention/instruction designed to meet their needs • http://www.ncpm.org

6. Progress Monitoring Supports a Dynamic Process for Individualizing Early Intervention • More Dynamic • More Data-Driven • More Responsive In this approach, children receive services sooner and more frequently when needed

7. Focus on Outcomes-Based Program Evaluation • Increased expectations for accountability • Need to identify children who need early intervention • Programs and individual staff members need to know when they are making a difference in moving children toward outcomes

8. Why are we interested in examining children’s outcomes? • To check on individual children’s growth • To inform parents about child growth • To ramp up individual children’s program if necessary • To get a check on how well our programs are doing • To focus our professional development efforts in our programs

9. Key Questions: • How can we learn more quickly that a child is falling behind in development? • How can we use that information to guide what we do in our programs? • How do we know if what we are doing is improving a child’s trajectory?

10. Most available approaches to program evaluation: • Conventional forms of early childhood assessment not linked to individual rates of growth toward outcomes • Child data not used in making ongoing program decisions • Not repeatable enough • Not accessible enough for program staff • Not sensitive to intervention or program effects or growth over time • Difficult for parents and practitioners to understand

11. What are Individual Growth and Development Indicators? • Measures that provide helpful information about children's growth toward socially valued outcomes and that guide intervention decision making • Measures that focus on key skills indicators rather than wide-band comprehensive skills

12. Individual Growth and Development Indicators • Individual • Growth and Development • Indicator

13. Helpful features of thermometer indicator: • Quick measure • Gives instant information • Tells a lot about an important general outcome: health • Can repeat it frequently

14. The most well known IGDIs are Pediatric Growth Charts Widely used by pediatricians and parents

15. Features of growth charts • Quick and easy to do • Relatively inexpensive • Repeatable • Can indicate potential problem and effectiveness of an intervention

16. Features of IGDIs Reflect progress toward a socially valid general outcome Strategic (a leading indicator) not comprehensive measurement Chart an individual’s progress Brief and quick to administer Repeatable (rate of growth, slope) Trend line compares expected vs. actual rates of learning

17. U.S. Survey of most important outcomes for ages birth to 8 years • Child uses gestures, sounds, words, or sentences to express wants and needs. • Child engages in a range of basic self‑help skills, including but not limited to skills in dressing, eating, toileting/hygiene and safety/identification. •  Child interacts with peers and adults, maintaining social interactions and participating socially in home, school, and community settings. •  Child manipulates toys, materials, and objects in a fluent and coordinated manner to play and participate in home, school, and community settings. • Child meets behavioral expectations (such as following directions, rules, and routines) in home, school, and community settings. -Priest et al, 2001

18. Early Communication Indicator (ECI) for Infants and Toddlers • The General Outcome • “The child uses gestures, sounds, words, or sentences to convey wants and needs or to express meaning to others." • In a national survey of parents and practitioners, expressive communication was a highly rated outcome of early childhood • Priest et al., 2001

19. Notice her different behaviors for expressing her wants and needs.

20. Early Communication Indicatorfor Infants and Toddlers • Key Skill Elements • Gestures • Vocalization • Single Words/Signs • Multiple Words/Signs • Combine to form Total Communication Indicator

21. Gestures Vocalizations Single Words Multi-Word Utterances Total Communication What Does the Expressive Communication IGDI measure? What does the IGDI Measure?

22. General Administration • Administration Procedures • Toy-play setting with familiar adult as play partner • 6-minute testing sessions • Play partner’s role is to facilitate play and follow child’s lead • Set-up/clean-up/put away • Alternate Toy Forms • Observational Recording Procedures Toy Form A: House Toy Form B: Barn

23. Example of Early Communication Indicator

24. Coding Sheet

25. Example of ECI Administration

26. An IGDI Example: Growth in Early Expressive Communication 36 Mos Expectation Child’s Observed Trajectory Normative Trajectory

27. Benefits/Strengths of IGDIs • GOMs not only show that children are acquiring skills, but they capture information about the rate of growth • Rate of growth can be compared to normative rates as well as to child’s own rate before or during an intervention or different phases or variations of intervention • This makes GOMs a sensitive way of looking at effectiveness of interventions

28. Trajectory or Rate of Growth is a Key Feature of GOM Approach to Progress Monitoring • Shows whether child’s current rate of growth is adequate for reaching the outcome. • Can easily communicate progress with family members, other professionals • Can indicate whether change in intervention is needed • Even when trajectory is far from typical, a positive trendline deflection in response to an intervention can convey when changes are “closing the gap”

29. IGDIs Currently Available • For Infants and Toddlers (http://www.igdi.ku.edu) • Early Communication (Language) • Early Problem Solving (Cognition) • Early Movement (Motor) • Early Social (Social/Emotional) • For Preschoolers (Early Literacy) (http://ggg.umn.edu/) • Picture naming (Spoken Vocabulary) • Alliteration • Rhyming

30. Results and how they are used-Charlie • Individual Child Level • Progress findings • Intervention decision making • Individual Program Level • Child Results • Implementation Quality and Staff Management • Project Level (e.g., State-wide) • Child Results • Implementation Quality and Staff Management

31. Individual Child Results • Scores Data Table • Total Communication Growth Chart

32. Individual Child Report Child’s Data Table Child’s Data Summary

33. Total Early Communication Graph Intervention or condition Line Child’s Scores Child’s Trajectory Slightly Below Benchmark Benchmark Trajectory Below Benchmark

34. Program-Level Results: Data Quality • Average Growth Trajectory • Individual Children of Concern • Distribution of Proficiency

35. Program Description: Average Growth

36. Targeting Individuals Children performing below benchmark. (Only available to program directors/coordinators.) Children due for an assessment

37. Distribution of Communication Proficiency

38. Project Level Results: Data Quality

39. Principles for developing measures-Charlie • Standards • Methods Used

40. Progress Monitoring Measurement Standards • The psychometric standards (AERA, 1999): • Reliability • Validity (Criterion and predictive) • Progress monitoring standards (NCPM, 2007): • Alternate forms • Sensitivity to student improvement (slope) • Annual yearly progress (AYP) benchmarks (level at end point) • Rates of improvement are specified (typical growth rates) • Improvement in teacher planning and/or student learning

41. Methods Used to Develop Measures • Literature Review and Synthesis • Draft General Outcome Definition • Identify Key Skills Identification, Selection, and Definitions Methods • National Survey • Socially Validate the General Outcome Definition • Pilot Testing • Identify and Select Toy Forms • Pilot Testing • Establish Administration Feasibility through User Testing • Cross Sectional Design Phase 1 Validation Study • Establish Sensitivity to Differences in Age; Reliability) • Longitudinal Design Phase 2 Validation Study • Demonstrate Sensitivity to Short-term Growth over Time; Reliability • Scale-up Application • Website Tools and Resources

42. Growth Trends in Early Communication Key Skills • Greenwood, C. R., Carta, J. J., Walker, D., Hughes, K., & Weathers, M. (2006). Preliminary investigations of the application of the Early Communication Indicator (ECI) for infants and toddlers. Journal of Early Intervention, 28(3), 178-196.

43. Concluding Points • GOM approach does offer reliable and valid tools for progress monitoring in early childhood. • They can be used for many purposes in intervention decision-making. • Their ease of use and illustration of growth make them excellent tools for communication about children’s progress. • Their sensitivity to growth allow for more frequent refinements and more effective interventions for individual children.

44. Advantages/limitations-Charlie • Advantages • Universal screening (all children) • Focus on short-term growth and development • Linked to response to intervention and prevention strategies • Supports staff planning and decisions regarding changing intervention • Designed for use by early interventionists and parents • Web-based support provides instant individual or group reports • Limitations • Requires a systematic effort to implement with fidelity • Requires administrative support to use at scale

45. Resources—websites-Charlie • For infants and toddlers • Communication, Movement, Problem Solving, Social, and Parent/Child Interaction • http://www.igdi.ku.edu • For preschoolers • Early Literacy: Vocabulary, Alliteration, and Rhyming • http://ggg.umn.edu • For kindergarten to grade 3 • Early Literacy: Reading • http://dibels.org

46. Infant and toddler Website • Public Pages: • Information, access, training, and materials • Private Pages (Requires UserID and Password): • Data services (collection, entry, management, and reporting) • $1 per child or • Participation in research/demonstration activities

47. Future—New progress measures for language/literacy • New progress monitoring work in development at the Center for Response to Intervention in Early Childhood (CRTIEC) • Oral Vocabulary • Phonemic Awareness • Book Knowledge • Comprehension • Online at http://www.crtiec.org

48. Selected References • Greenwood, C. R., Carta, J. J., & Walker, D. (2005). Individual growth and development indicators (IGDIs): Tools for assessing intervention results for infants and toddlers. In B. Heward & et al. (Eds.), Focus on Behavior Analysis in Education: Achievements, Challenges, and Opportunities (Chapter 6) (pp. 103-124). Columbus, OH: Pearson/Prentice-Hall. • Greenwood, C. R., Carta, J. J., Baggett, K., Buzhardt, J., Walker, D., & Terry, B. (2008). Best practices in integrating progress monitoring and response-to-intervention concepts into early childhood systems. In A. Thomas, J. Grimes & J. Gruba (Eds.), Best practices in school psychology V (pp. 535-548). Washington DC: National Association of School Psychology. • McConnell, S. R., McEvoy, M. A., & Priest, J. S. (2002). Growing measures for monitoring progress in early childhood education: A research and development process for Individual Growth and Development Indicators. Assessment for Effective Intervention, 27(4), 3-14. • Walker, D., Carta, J. J., Greenwood, C., & Buzhardt, J. (2008). The use of Individual Growth and Development Indicators for progress monitoring and intervention decision making in early education. Exceptionality, 16(1), 33-47.

49. Selected References • Carta, J. J., Greenwood, C. R., Walker, D., Kaminski, R., Good, R., McConnell, S. R., & McEvoy, M. (2005). Individual growth and development indicators (IGDIs): Assessment that guides intervention for young children. Young Exceptional Children, 4, 15-27. • Greenwood, C. R., Carta, J. J., & Walker, D. (2004). Individual growth and development indicators (IGDIs): Tools for assessing intervention results for infants and toddlers. In B. Heward et al., (Eds.), Focus on Behavior Analysis in Education: Achievements, Challenges, and Opportunities (Chapter 6, pp. 103-124). Pearson/Prentice-Hall: Columbus, OH. • Greenwood, C. R., Carta, Walker, D., Carta, J. J., & Hughes, K. (2006). Preliminary investigators of the application of the Early Communication Indicator (ECI) for infants and toddlers. Journal of Early Intervention, 28, 178-196. • Greenwood, C. R., Carta, J. J., Baggett, K., Buzhardt, J., Walker, D., & Terry, B. (2008). Best practices integrating progress monitoring and response to intervention concepts into early childhood systems for infants and toddlers. In A. Thomas, J. Grimes & J. Gruba (Eds.), Best practices in school psychology V. Washington DC: National Association of School Psychology, Washington, DC.

50. But trajectories can illustrate whether child is on a path toward achieving important outcomes or is closing the gap. If you don’t know where you’re going, any road will get you there. ----Lewis Carroll