Use of the NCSEAM Preschool and Part C Surveys to Address the SPP/APR Parent/Family Indicators

1. Use of the NCSEAM Preschool and Part C Surveys to Address the SPP/APR Parent/Family Indicators Batya Elbaum, Ph.D. University of Miami National Center for Special Education Accountability Monitoring National OSEP Early Childhood Conference Washington, D.C. December 13, 2005

2. SPP/APR Part C Indicator “Percent of families participating in Part C who report that early intervention services have helped the family a) know their rights, b) effectively communicate their children’s needs, and c) help their children develop and learn.”

3. SPP/APR Part B Indicator “Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities.”

4. Steps in addressing the SPP/APR parent/family indicators • Select a measurement tool • Establish a sampling plan • Select mode of administration • Work out details of the data collection process • Collect and analyze data • Create reports

5. NCSEAM Parent/Family Involvement Workgroup • Began work in January 2002 to develop a set of valid, reliable, and useful measures of families’ perceptions and involvement in early intervention and special education.

6. Current status of the NCSEAM Parent/Family Measures • The NCSEAM survey instruments for Part C and Part B school-age were finalized in August 2005 • The NCSEAM survey instrument for 619 will be finalized by February 2006

7. Item development • Items were drawn from: • existing survey instruments • descriptions of best practices in parent involvement and family-centered services • experts in the field • Stakeholder workgroups were conducted in 6 states (MS, NH, CA, NM, KY, FL) •  500 suggested items were reviewed by PACER and other parent groups

8. Item validation • The NCSEAM National Item Validation Study was conducted between October 2004 and February 2005 • The Part C sample consisted of ~ 1750 families from 8 states • The Part B sample consisted of ~ 2500 parents from 6 states, including 71 parents of preschool children • Different but linked item sets were used for Part B and Part C

9. Further development of items for 619 • A 619 workgroup was convened in conjunction with the February 2005 OSEP Early Childhood Conference • Participants suggested additional items and wording changes to ensure that items were appropriate for parents of preschool children • Feedback was solicited via the 619 listserv in spring 2005 • Overall, input was provided by 619 staff from over 25 states

10. Further development of items for 619 • Additional feedback was provided by the NCSEAM hParent/Family Involvement Workgroup and a local 619 parent-professional team • Data from Part B parents who participated in the National Item Validation Study were analyzed separately for parents of children K-12 and parents of children receiving preschool special education services

11. What do the items measure? • Items included in the NCSEAM National Item Validation Study are not all “about” one thing. • They reflect several different dimensions of parents’ perceptions and involvement. • Each dimension is measured by its own set of items. • Each separate set of items constitutes a scale.

12. What are the dimensions? • Parents’ perceptions of school/program efforts to engage them in meaningful partnerships • Parents’ perceptions of the quality of services provided to their child and family • Parents’ reports of their participation in the early intervention/special education process • Parents’ perceptions of the impact of services on their family • Parents’ perceptions of the impact of services on their child*

13. Scales for Early Intervention • Parents’ perceptions of programs’ partnership efforts and quality of services • Parents’ perceptions of the impact of services on their family

14. Scales for 619/preschool • Parents’ perceptions of school/program partnership efforts and quality of services • Parents’ perceptions of the impact of services on their family • Parents’ reports of their participation in the preschool special education process

15. Scales for Part B school-age • Parents’ perceptions of school partnership efforts • Parents’ perceptions of the quality of services provided to their child • Parents’ perceptions of the impact of services on their family • Parents’ reports of their participation in the special education process

16. Major finding from the National Item Validation Study • Within each scale, the items showed virtually the same order of agreeability for all respondents. • The consistent ordering of items in a scale is the foundation of good measurement.

17. An example of scale invariance: Dr. B’s Running Fitness Survey

18. Confirmation of Invariance • Across alternate forms • Across subgroups defined by: • State • ethnicity • age of child • language in which the survey was administered (English vs. Spanish) • method of administration (self-administered or items read by a facilitator) • Across C, 619, and B school-age (for items with similar content)

19. Sample 619 Efforts/Quality Items • People from preschool special education, including teachers and other service providers: • provide me with information in a language I understand. • give me enough opportunities to discuss my child’s progress. • encourage me to participate in the decision-making process.

20. Sample 619 Efforts/Quality Items • We discussed special help my child would get so he/she could learn with children without disabilities. • People from preschool special education are good at working with families. • I am considered an equal partner in planning my child’s preschool special education.

21. Sample 619 Efforts/Quality Items • People from preschool special education, including teachers and other service providers: • provide services to my child in a timely manner. • give me information about the instructional methods used with my child. • Someone explained to me what options parents have if they disagree with a decision made by people from the preschool special education program.

22. Sample 619 Efforts/Quality Items • People from preschool special education, including teachers and other service providers, give me options concerning my child’s services and supports. • I was offered special assistance (e.g., child care or transportation) so that I could participate in the IEP/IFSP meeting. • People from preschool special education connect families with organizations that serve parents of children with disabilities.

23. Sample Part C Impact on Family Items Over the past year, early intervention services have helped me and/or my family: • Understand my child's special needs. • Do things with and for my child that are good for my child's development. • Be more optimistic about my child's future. • Feel that my efforts are helping my child.

24. Sample Part C Impact on Family Items Over the past year, early intervention services have helped me and/or my family: • Communicate more effectively with the people who work with my child and my family. • Feel more confident in my skills as a parent. • Understand the roles and responsibilities of the people who work with my child and family.

25. Sample Part C Impact on Family Items Over the past year, early intervention services have helped me and/or my family: • Know about my child's and family's rights concerning special education services. • Know where to go for help or support to meet my child's needs. • Make changes in our family routines that will benefit my child with special needs.

26. Sample Part C Impact on Family Items • Over the past year, early intervention services have helped me and/or my family: • Participate in typical activities for children and families in my community. • Understand how the special education system works. • Know about services in the community.

27. ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS RULER* • Theschool offers parents training . . . [660] • The school explains what options parents have . . . [600] • Written information I receive is . . . understandable. [520] *.95 likelihood of agreement

28. . .##### . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .######## PARENT MEASURES ON THE PART B SCHOOL EFFORTS RULER In the histogram to the left of the ruler, each # sign represents 13 parents (total n=2634). In the national sample, the average parent measure on the Part B School Efforts scale was 500. The parent measures ranged from a low of 140 to a high of 880.

29. Steps in the NCSEAM standard-setting process • In July 2005, NCSEAM convened a stakeholder group with broad representation of families, state and local agencies, advocates, and researchers. • Participants were provided with the list of scale items in calibration (agreeability) order.

30. Steps in the NCSEAM standard-setting process • The question posed to the group was: What is the highest item with which you would require an “agree” response in order to have confidence that the meaning of the indicator (i.e., schools are facilitating parent involvement) is being achieved?

31. Steps in the NCSEAM standard-setting process • Participants had to reach consensus on this question. • The point on the ruler that corresponds to the selected item represents the standard. • “If families don’t agree with this item” – and, by implication, with all those below it – “then we could not say that we had acceptable quality in this area.”

32. Applying the standard • Performance on the indicator is calculated as the percent of parents or families with measures at or above the established standard.

33. PART B SCHOOL PARTNERSHIP EFFORTS • S offers parents training about special education issues • S gives parents help to play an active role in child's education • I organizations that offer information and training for parents • I have been asked for my opinion about services • I was offered special assistance so I could participate in IEP • The school provides information on agencies for transition • S gives me choices with regard to services • S explains what options parents have if they disagree with school • S communicates regularly with me regarding my child's progress • We discussed how my child would participate in assessments • Written justification given for services outside regular class • Teachers and administrators seek out parent input • T&A ensure that I have fully understood the Procedural Safeguards • I am considered an equal partner with teachers and professionals • All of my concerns and recommendations were documented on the IEP • T&A show sensitivity to the needs of students with disabilities • S offers parents different ways to communicate with teachers • T&A encourage me to participate in the decision-making process • My child's evaluation report is written in terms I understand • We discussed accommodations and modifications for my child • T treat me as a team member • T&A respect my culture and language • Written information I receive is written in an understandable way • T are available to speak with me • Information is provided to me in a language I understand

34. ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS RULER* . .##### . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .######## 24% • The school explains what options parents have if they disagree with a decision of the school. [600] *.95 likelihood of agreement

35. . . . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .######## ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS RULER* 30% • The school explains what options parents have if they disagree with a decision of the school. [600] *.95 likelihood of agreement

36. PART C IMPACT ON FAMILY • Over the past year, Early Intervention services have helped me and/or my family: • participate in typical activities for children and families in my community • understand how the Early Intervention system works. • keep up friendships for my child and family. • know about services in the community. • know where to go for support to meet my family's needs. • be more effective in managing my child's behavior. • know about my child's and family's rights concerning early intervention services. • improve my family's quality of life. • know where to go for support to meet my child's needs. • make changes in family routines that will benefit my child with special needs. • do activities that are good for my child even in times of stress. • get the services that my child and family need. • feel that my family will be accepted and welcomed in the community. • communicate more effectively with people who work with my child and my family. • feel that my child will be accepted and welcomed in the community. • be able to evaluate how much progress my child is making. • feel more confident in my skills as a parent. • understand the roles of the people who work with my child and family. • feel that I can get the services and supports that my child and family need. • understand my child's special needs. • do things with and for my child that are good for my child's development. • feel that my efforts are helping my child.

37. ITEM CALIBRATIONS ON THE PART C FAMILY OUTCOMES RULER* .############### .## .# .# .## #### .### .#### .#### .#### .######## ###### .####### .##### .##### .### .##### .#### .#### .###### ######### .### .## .## .# .# .# .# . . . . . . . . . .# Over the past year, Early Intervention services have helped me and/or my family: • know about my child’s and family’s rights . . . [600] • communicate more effectively . . . [570] • do things with and for my child that are good for my child's development [540] *.95 likelihood of agreement

38. Constructing smaller item sets • To reproduce the measures obtained using all the items in a scale, subsets of items need to represent the full range of the measurement ruler. • With appropriate selection, different subsets of items will produce the same measures. • Approximately 25 items per scale are needed to achieve measurement reliability of .90 or above.

39. How the NCSEAM measures can guide improvement efforts • A state’s measure on the indicator can be directly interpreted with respect to the items that define the scale. • Items below the measure are in large part being accomplished; items above the measure have yet to be accomplished. • Items just above the measure are good targets for improvement.

40. C -> 619 -> B linkages Example #1: Facilitating parent networking • Part C: Someone from the early intervention program helped me get in touch with other parents for help and support. • 619: People from preschool special education, including teachers and other service providers, connect families with one another for mutual support. • Part B school-age: I was given information about organizations that offer support for parents of students with disabilities

41. C -> 619 -> B linkages Example #2: Ensuring that parents know their rights • Part C: My family was given information about the rights of parents regarding early intervention services. • 619: People from preschool special education, including teachers and other service providers, ensure that I have fully understood my rights in preschool special education. • Part B school-age: Teachers and administrators ensure that I have fully understood the procedural safeguards (the rules in federal law that protect the rights of parents).

42. Use of multiple scales • The SPP/APR family indicator for Part C can be addressed by data derived from a single scale, namely, the Impact on Family Scale • The SPP/APR parent indicator for 619 can be addressed by data derived from a single scale, namely, the Efforts/ Quality Scale • However, use of additional scales can provide valuable continuity of measurement as well as inform program improvement.

43. Further information • Detailed information on the NCSEAM parent surveys is currently posted to the website of the Federal Resource Center for Special Education: www.dssc.org/frc/