Kathleen Baggett, Ph.D. Kathryn Bigelow, Ph.D. Prepared for the Head Start Center for Inclusion

1. Embedding Progress Monitoring in Interventions to Support Social-Emotional and Communication Development I Kathleen Baggett, Ph.D. Kathryn Bigelow, Ph.D. Prepared for the Head Start Center for Inclusion November 3, 2010

2. Session Goals • Explore the relevance and importance of progress monitoring for improving child outcomes in EHS • Provide a brief overview of progress monitoring • Consider some examples of progress monitoring tools and their application to interventions • Stimulate discussion about how progress monitoring can be applied within your work

3. What is Progress Monitoring... and why is it important? • Scientifically-based practice used for assessing child progress and thus, intervention effectiveness • Allows for documentation of progress and gives us the ability to adjust the intervention to meet intervention goals • Provides data on intervention effectiveness that can be provided to a wide audience (administrators, parents, researchers, funders) Adapted from National Center on Student Progress Monitoring

4. Progress Monitoring • Measures should occur on a regular, frequent basis • Should be brief and easy to administer • Should be sensitive to the intervention after relatively short periods of time • Should provide information that is readily understandable to a range of individuals (administrators, parents, etc) • Should allow for monitoring progress of individual children, and groups of children Adapted from National Center on Student Progress Monitoring

5. What is Fidelity Monitoring...and why is it important? • Documentation of the degree to which an intervention is implemented as originally planned • Without fidelity, we do not know if our results, good or bad, are a result of the intervention • Tells us where efforts to increase adherence to the intervention protocol are needed • Crucial to determining if an intervention is effective

6. Fidelity Monitoring • Measures must be closely tied to intervention protocol • Measures should be based on a clear and objective description of the steps involved in the intervention • After observations, calculate the percentage of accurate implementation, provide feedback, and take steps to increase accuracy, if necessary • Measurement should take place on an on-going basis, as implementation can change across providers or over time

7. IPCI A General Outcomes Measure: For checking growth in caregivers’ responsiveness to their child in ways that promote positive social-emotional behavior.

8. EHS Home Visitors Home Visiting Nurses Parent-Child Interaction Part C EI Home Visitors Mental Health Therapists IPCI

9. Practitioners need practical tools that can: Provide quick snapshots of parent-child interaction that can be taken repeatedly to: • Help interventionists know when intervention is needed • Help interventionists see when they are making a difference • Help interventionists know when an intervention change is needed • Help supervisors facilitate intervention-planning • Help program directors understand when programs need improvement • While there are many measures of parent-child interaction, tools designed specifically for practitioners to guide intervention decision-making have been lacking.

10. IPCI Purpose Monitor progress Identify risky interactions Guide intervention decision-making

11. Progress Monitoring for all children If cautions are identified, increase monitoring If concerns are identified, Begin intervention Decision Making Model for Generating Options

12. IPCI Administration Administered in family homes or other caregiving settings (biological home, foster home, center-based care) Children 3-42 months of age and a familiar caregiver Four semi-structured activities are observed for a total of 10 minutes Free Play Looking at Books Distraction Task Dressing 12 items are rated on a 4-point scale following observation Videotaping is not required (but useful for intervention purposes)

13. IPCI Activities Looking at Books Free play Distraction Dressing

14. IPCI Activities Free play (4 minutes) “Whatever it is that you and your child like to do together- something your child loves”

15. IPCI Activities Looking at Books (2 minutes) “Here are some books for you and your child to look at together- however you and your child would like to spend time with these books is fine.”

16. IPCI Activities Distraction Task (2 minutes) “There are times when parents need to keep children away from things- either because they may be dangerous or just inappropriate”. Let’s see how your child responds when there are materials that are off limits. Please keep child on the blanket and away from the recorder and keys”.

17. IPCI Activities Dressing (2 minutes) “Whatever it’s like to get dressed in the morning” (shirt, socks, shoes)

18. Caregiver Facilitators Conveys acceptance and warmth Makes descriptive comments Follows child’s lead Maintains and extends Caregiver Interrupters Uses criticism, harsh tone Uses intrusions and restrictions Child Engagement Positive feedback Sustained engagement Follow-through Child Distress Fuss, cry Tantrum Frozen, watchful, withdrawn IPCI Domains and Behaviors

19. Caregiver and Child Behaviors are rated on a 4-point scale O = Not at all 1 = Rarely 2 = Sometimes but inconsistently 3 = Often and consistently Behavior Ratings

20. The IPCI’s Home: • With other early childhood general outcomes measures • Early Communication Indicator • Early Social Indicator • Early Problem Solving Indicator • Early Motor Indicator

22. Types of IPCI Reports • Home Visitors and Supervisors • Program Administrator • Agency Administrator

27. Kansas Model Demonstration Center • Goal of project is to support early intervention providers in using evidence-based practices to help build the capacity of parents and educators in promoting communication of young children • Fidelity and progress monitoring include documentation of: • Parent and teacher use of communication strategies to monitor intervention progress and fidelity and provide feedback , and child communication • Parent and teacher self-report of communication strategies to monitor progress, provide feedback, and prompt use of strategies • Fidelity of implementation of communication strategies by providers, who are delivering intervention to parents and teachers

28. Monitoring Parent/Teacher Strategy Use and Child Communication • Parent and teacher use of communication promoting strategies measured using a frequency count of strategy use during 30-min observations • Also measure child communication (gestures, vocalizations, words, multiple words) • Providers share graphs with parents and teachers monthly to provide feedback on use of strategies, illustrate connection between adult and child communication, and show growth in child communication over time • Provider uses this as an opportunity to provide positive feedback, address challenges, and prompt further use of strategies throughout routines

29. Parent/Teacher Use of Strategies and Child Communication Dotted line shows frequency of use of communication promoting strategies by a parent, and bars show child communication

30. Parent and Teacher Self-Report Use of Communication Strategies • Parent and teacher complete self-checks of their use of communication promoting strategies during a variety of routines • Rate frequency with “not today”, “rarely”, “sometimes”, and “often” • Providers collect forms and use data to prompt discussion with parent/teacher about use of the strategies, provide positive feedback, and share additional ideas for embedding strategies in new routines or activities • Functions as a prompt to remember to use strategies, but also allows parents or teachers to self-monitor their use of strategies

34. Measure of Fidelity of Implementation: Providers • Simple checklist used to document the strategies shared with parents/teachers, strategies observed during visits, and routines in which strategy use was addressed • Providers fill this out themselves after each visit and regular reliability checks are completed by intervention coach • Data are summarized and a report is provided monthly, summarizing data for each family • Intervention coach distributes reports to providers in meetings or in person, and posts on password-protected web site • Coaches use this as an opportunity to discuss provider’s strategies for sharing intervention with parents/teachers, provide positive feedback, address challenges, and plan next steps in intervention

35. Juniper Gardens MDC Communication Strategies □ Environ. Arrangement □ Following child’s lead □ Commenting/labeling □ Imitating/expanding □ Open-ended questions □ Positive attention/praise □ Providing choices □ Fill in blank/time delay Strategies discussed or taught in: □ Play □ Meal/snack time □ Toilet/personal care □ Household routines □ Reading/ books □ Circle/Group □ Outings □ Outdoor □ Other (desc): ________ Observed parent using strategies in: □ Play □ Meal/snack time □ Toilet/personal care □ Household routines □ Reading/ books □ Circle/Group □ Outings □ Outdoor □ Other (desc): ________ Discussed Manual/DVD? □ Yes □ No Reviewed Self-Checklist? □ Yes □ No □ N/A Discussed Graph? □ Yes □ No □ N/A

36. Juniper Gardens MDC Provider _____ Child ________ Date ___ / ___ / ___ Worked with (check all that apply): □ Child (directly) □ Parent □ Teacher Language is the Key Strategies used, taught or discussed: □ Comment and Wait □ Ask Questions and Wait □ Respond by Adding More (expansion) □ Following Child’s Lead □ Other (desc): __________________ Strategies discussed or taught During: □ Play □ Reading/ books □ Daily Activities □ Other ____________ Observed parent/teacher using strategies during: □ Play □ Reading/ books □ Daily Activities □ Other ____________ Introduced Lang is the Key Manual/DVD for first time? □ Yes □ No □ n/a Reviewed Self-Checklist? □ Yes □ No Reviewed Self-Checklist? □ Yes □ No Discussed Graph? □ Yes □ No □ n/a

37. Provider Summary Report showing strategies addressed with a given family so far, activities in which strategies were addressed, and strategies observed throughout the visits.

38. Fidelity and Progress Monitoring Working Together • Fidelity monitoring through observations of parents and teachers, and of early intervention providers offers a wealth of information on whether intervention is being implemented as planned • Fidelity monitoring through observations of parents, teachers, and providers tells us the degree to which intervention is being implemented as originally planned (and initially evaluated) • Progress monitoring shows us how parents and teachers are using strategies and how child communication is growing and how social-emotional behavior is changing • Provides greater confidence in effectiveness of the intervention

39. References and Resources • Carta, J., Greenwood, C., Walker, D., & Buzhardt, J. (Eds.). Individual Growth and Developmental Indicators: Tools for Monitoring Progress and Measuring Growth in Very Young Children. Baltimore, MD: Brookes. • IGDI website: http://www.igdi.ku.edu/ • National Center on Student Progress Monitoring – www.studentprogress.org