Collecting Meaningful Parent and Family Involvement Data You Can Use

1. Collecting Meaningful Parent andFamily Involvement Data You CanUse Amy Bitterman, IDEA Data Center Hadley Moore, IDEA DataCenter Carol Seay, Georgia Department ofEducation 2019 OSEP LEADERSHIPCONFERENCE

2. 2019 OSEPLeadership Conference DISCLAIMER: The contents of this presentation were developed by the presenters for the 2019 OSEP Leadership Conference. However, these contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the FederalGovernment. (Authority: 20 U.S.C. 1221e-3 and3474) 2

3. What Will We Do in Today’sSession? • Present an overview of State Performance Plan (SPP)/Annual Performance Report (APR) Indicators B8 andC4 • Describe approaches for measuring and improving the representativeness, or meaningfulness, of parent and family involvementdata • Explore strategies for using parent and family involvement data, beyond the requirements of theindicators 2019 OSEP LEADERSHIPCONFERENCE

4. The Parent and Family APRIndicators • Indicator B8: The percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children withdisabilities • Indicator C4: The percent of families participating in Part C who report that early intervention services have helped thefamily • Know theirrights • Effectively communicatetheir children’sneeds • Help their children develop andlearn 2019 OSEP LEADERSHIPCONFERENCE

5. What Are StatesDoing? • All states usedsurveys • Part B – 48% used state-developed surveys and 39% used the National Center for Special Education Accountability Monitoring (NCSEAM)survey • Part C – 46% used the Early Childhood Outcomes Family Outcomes Survey, 32% used the NCSEAM survey, and 21% used state-developedsurveys Sources:2017PartBandPartCFFY2015SPP/APRIndicatorAnalysisBookletandIDC’s surveyofstates 2019 OSEP LEADERSHIPCONFERENCE

6. What Are States Doing in PartB? • 53% of states usedsampling • About a third of the states had the districts or schools distribute thesurveys • Most states administered the survey via paper andweb Source: IDC’s survey ofstates 2019 OSEP LEADERSHIPCONFERENCE

7. What Are States Doing in Part B?(cont.) • A third of the states distributed notifications alerting parents to the upcoming survey • Field periods ranged from 6 weeks to 9months • About half of the states sent reminders to parents who had not completed thesurvey • Many states’ response rates were below30% Source: IDC’s survey ofstates 2019 OSEP LEADERSHIPCONFERENCE

8. Parent Survey inGeorgia • Currently in the 3rd year of an onlinesurvey • Available to all families of children withdisabilities • 10 questions plus demographic dataquestions • 6 point LikertScale • Local education agencies (LEAs) receive an overall percentage measuring parentalinvolvement • The “power” of the data is the availability of real time school-leveldata • Special EducationDashboard • Data is available to special education directors daily, January –May • School-level data 2019 OSEP LEADERSHIPCONFERENCE

9. Parent Survey Application, SpecialEducation Dashboard Slide 9 When the surveyis open this updates nightly 2019 OSEP LEADERSHIP CONFERENCE

10. Parent Survey – DownloadableData • Demographicdata • Redacted from the download data to avoid personally identifiable information (PII) • School Level • Participation by school • Updatednightly • By question – Strongly agree….stronglydisagree • Real-time data to answerquestions • What schools had/have high participation?Why? • What schools had/have parents who report that schools encourage their involvement? • Where is workneeded? • Examine responses question byquestion 2019 OSEP LEADERSHIPCONFERENCE

11. PreviousSurvey • Georgia’s old survey, paper with an onlineoption • Sampling • Some smaller LEAs were on a 5-yearcycle • Not all parents were given the opportunity to participate; some never had opportunity • Far fewer surveyscompleted • Results forLEAs • A singlepercentage • Ex.:49% • Not useful – LEAs only looked to see if they met thetarget • Results were never shared with schooladministrators • Results were not available by school; a single percentage wasmeaningless 2019 OSEP LEADERSHIPCONFERENCE

12. NewSurvey • LEAs continue to receive an overall percentage measuring parental involvement and this is compared to the target – Indicator 8data • Example – FY18 Georgia’s target was69% • If an LEA’s overall percentage was at or above 69%, the LEA met the target • The survey measures parental involvement data by calculating the mean for each parent based on his or her responses to the 10 questions • Parents whose responses yield a mean of 5 are included in the numerator; all parents responding are included in thedenominator 2019 OSEP LEADERSHIPCONFERENCE

13. Calculating thePercentage Parent 1 responses: Q1: Very strongly agree;6 Q2: Strongly agree;5 Q3: Very strongly agree;6 Q4: Agree;4 Q5: Strongly agree;5 Q6: Very strongly agree;6 Q7: Agree;4 Q8: Disagree;3 Q9: Very strongly agree;6 Q10: Very strongly agree;6 Total points for Parent1: 51 10 questions; 10/51 = 5.1 This parent has a meanof ≥5 and will beincluded when calculating parental involvement percentage 2019 OSEP LEADERSHIPCONFERENCE

14. NewSurvey • In addition to an overall LEA percentage, LEAsreceive • Responses by disabilityarea • Responses byrace/ethnicity • Responses by gradeband • An item ranking – highest to lowest rankeditems • Real benefit is school-leveldata 2019 OSEP LEADERSHIPCONFERENCE

15. Measuring and Improving Representativeness of Parent and Family InvolvementData “Representativeness” is the extent to which the demographics of the children whose parents participated in data collection activities are representative of the demographics of all children receiving special education/early intervention services in thestate 2019 OSEP LEADERSHIPCONFERENCE

16. Why Is RepresentativenessImportant? • Representativeness affects the meaningfulness of yourdata • If your data are not representative, they will not paint an accurate picture of the nature and level of parent and family involvement in yourstate 2019 OSEP LEADERSHIPCONFERENCE

17. Preparing to Collect RepresentativeData • How states administer surveys can affect data quality – especially response rates andrepresentativeness • Instrumentdesign • Sampling • Survey mode and distributionmethod • Access to or collection of demographicdata • Trackingresponses 2019 OSEP LEADERSHIPCONFERENCE

18. Measuring Representativeness: What Have States BeenDoing? States commonly assessed representativeness by comparing the characteristics of the children whose parents responded to the characteristics of the entire state population of special education or early interventionchildren Twoor MoreRaces 0% 2% 82% White 48% NativeHawaiianorOtherPacificIslander 0% 1% 16% Black or AfricanAmerican 38% Asian 0% 2% AmericanIndianor Alaska Native 0% 1% Hispanic/Latino 2% 10% 0%10%20%30%40%50%60%70%80%90%100% Students whoseparentsresponded All students withdisabilities Source: 2017 Part B FFY 2015SPP/APR Indicator Analysis Booklet and IDC’s survey ofstates 2019 OSEP LEADERSHIPCONFERENCE

19. How Can States AssessRepresentativeness? • Compare the characteristics of children whose parents complete the survey with those of all children with disabilities or compare the characteristics of children of responders to children ofnonresponders • Examine multiple characteristics – child’s race/ethnicity, sex, disability, age, geographiclocation • Obtain survey responses from a sample of parents who initially did not respond to see if the way they respond to the survey differs from the responses you alreadyhave 2019 OSEP LEADERSHIPCONFERENCE

20. How Can States AssessRepresentativeness? (cont.) • No one threshold for determining whether data arerepresentative • Tools for calculatingrepresentativeness • The National Post-School Outcomes Center (NPSO) Response Calculator uses a threshold of ±3 percent in observeddata • The Early Childhood Technical Assistance Center (ECTA) Response Rate and Representativeness Calculator uses tests of statistically significant differences to determinerepresentativeness 2019 OSEP LEADERSHIPCONFERENCE

21. How Can States ImproveRepresentativeness During DataCollection? • Conduct periodic checks forrepresentativeness • Send additional reminders to parents who have the specific characteristics that areunderrepresented • Enlist support from Parent Centers or other key stakeholder groups to encourage parents from underrepresented groups torespond • Call parents who started but did not complete the survey to encourage them torespond 2019 OSEP LEADERSHIPCONFERENCE

22. How Can States Improve RepresentativenessAfter Data Collection and in Future DataCollections? • Weight data to account for nonrepresentativedata • In the future, include larger subsamples of parents from underrepresented groups (called“oversampling”) • Work with Parent Centers and other stakeholders with connections to underrepresented groups of families to plan activities to improve representativeness in the next datacollection 2019 OSEP LEADERSHIPCONFERENCE

23. How Georgia Measures and ImprovesRepresentativeness 2019 OSEP LEADERSHIPCONFERENCE 23

24. GA FY18 Representativeness – DisabilityArea FY18 Parent Survey, Representativeness by DisabilityArea 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% 35.0% 27.5% 15.4% 14.8% 14.7% 13.6% 11.5% 10.2%11.7% FY18: Percentage of Responses by Disability Area of Children whose ParentsResponded FY18: % of Students by DisabilityArea 9.7% 7.5%7.8% 5.3%5.0% 0.5%0.4% 2019 OSEP LEADERSHIPCONFERENCE

25. GA FY18 Representativeness –Race/ethnicity Indicator 8: FY18 ParentSurvey 60.0% 53.8% 50.0% 40.5% 39.4% 40.0% FY18: Percentage of Responses by Race/Ethnicity of Children whose Parents Responded FY18: StudentDemographics 27.8% 30.0% 20.0% 14.3% 10.6% 10.0% 3.8%3.5% 2.0% 2.0% 1.4%0.0% 0.5%0.2% 0.0% White Black Hispanic Asianor Pacific Islander American Indian or Alaska Native Mulit-racialUnknown 2019 OSEP LEADERSHIPCONFERENCE

26. AddressingRepresentativeness • Important to note that Georgia offers all families the opportunity to participate in the survey; surveys are not distributed to selectfamilies • Increase overallparticipation • Share results with district and school leaders and teachers and enlistsupport • – real time (during the survey open period) and finalresults • Publicize the survey – newsletters, emails,website • Provide technology at meetings, conferences,events • Analyze real-timedata • State encourages special education directors to examine their data throughout the survey period to addressrepresentativeness 2019 OSEP LEADERSHIPCONFERENCE

27. Point-in-TimeData Number of Completed Parent Surveys, SampleLEA 120 109 100 80 66 60 43 40 36 40 30 18 20 13 10 3 Happy Sunny Cloudy Elementary Elementary Elementary School School School 0 Town High City Middle School School Country East North Elementary Elementary Elementary School School School WestHigh School South Middle School 2019 OSEP LEADERSHIPCONFERENCE

28. ParentMentorPartnership (PMP) • The Georgia Parent Mentor Partnership is parents and professionals working together to improve outcomes for students with disabilities by enhancing communication and collaboration between families, educators and the community. • Georgia is proud of the work we are doing to engagefamilies • Parent Mentors play a key role in increasingparticipation • Georgia is working to give families a voice and encourage teaming with schools to improve outcomes forchildren 2019 OSEP LEADERSHIPCONFERENCE

29. Discussion:Representativeness • How have you seen states examine the representativeness of parent and family involvementdata? • What strategies have states used to improverepresentativeness? • What works well and what doesn’twork? 2019 OSEP LEADERSHIPCONFERENCE

30. Using Parent and Family Involvement Data,Beyond theIndicators: Slide 30 Let's explore the “why” and the “how” of using parent and family involvement data beyond theindicators! 2019 OSEP LEADERSHIP CONFERENCE

31. Make the Most of Parent andFamily InvolvementData • Why? • Ensure you are providing high-qualityservices • Promote accountability • Identify and prioritize needs forimprovement • Create and support family engagementinitiatives • How? • Go beyond the indicator • Add qualitativefindings • Involvestakeholders • Combine with otherdata • Share findings 2019 OSEP LEADERSHIPCONFERENCE

32. Go Beyond the Indicator to MakeMeaning • Generate descriptivestatistics • Compare performance to investigate patterns or differencesacross • Districts andprograms • Years • Childcharacteristics • Surveyitems • Explorehypotheses • Link back to questions you wantanswered 2019 OSEP LEADERSHIPCONFERENCE

33. Add Qualitative Findings to ProvideDepth • Open-ended items on a Parent or Family Involvement survey are an example of qualitative data that you might already becollecting • Inclusion of qualitative findings can provide greater understandingof • Factors associated with parent or familyinvolvement • Parent or family perceptions • Barriers to achieving greater parent or familyinvolvement 2019 OSEP LEADERSHIPCONFERENCE

34. Involve Stakeholders to GainInsight • Stakeholders can helpyou • Quickly identifypatterns • Understand the meaning behind certainfindings • Explore differenthypotheses • Feel confident you understand the true meaning of yourdata • And, continue to involve stakeholdersin • Decisionmakingprocesses • Developing overall program priorities and improvementstrategies 2019 OSEP LEADERSHIPCONFERENCE

35. Combine With OtherData • Demographicdata • Title I Parent and Family Engagementdata • State Systemic Improvement Plan (SSIP)data • Child outcomesdata • Graduation, drop out, and academic achievementdata 2019 OSEP LEADERSHIPCONFERENCE

36. Share Findings With VariousAudiences • Sharing findings is critical to make the most of your efforts and effect change • Considerations for sharing findings with familiesinclude • Share information that is highly relevant tothem • Focus on plans for improvement • Ensure communications are accessible to allfamilies 2019 OSEP LEADERSHIPCONFERENCE

37. Share Findings With Various Audiences(cont.) • Mechanisms for sharing witheducators • Regular communications, such as newsletters oremails • Committeemeetings • Internal local dashboards for districts • Statewide meetings (have districts with improved or high response rates share successfulstrategies) 2019 OSEP LEADERSHIPCONFERENCE

38. How Georgia Makes the Mostof Data 2019 OSEP LEADERSHIPCONFERENCE 38

39. Using the Parent Survey Data inGeorgia • State Level • Family Engagement and Dispute Resolution Unit at theGADOE • Special Education Family Engagement staff are part of thisunit • Coupling family engagement and the ‘Help Desk’ work with the investigation of state complaints isintentional • Georgia is exploring ways to correlate the Indicator 8 data with the dispute resolutiondata • Data is useful in otherways • State staff leads the work for using Parent Survey Data at the LEA and School Level 2019 OSEP LEADERSHIPCONFERENCE

40. How Georgia Uses theData • More than federalreporting • The usefuldata • Schoollevel • Share with schoolleaders • Share with teachers • Question-by-question analysis ispossible • Target efforts to schools and specific areas ofneed • Ultimategoal • Know and understand where and why families believe they are considered to be a true partner in improving outcomes for theirchild(ren) • Address the issues at the schools where families do not believe they are considered to be apartner 2019 OSEP LEADERSHIPCONFERENCE

41. Example1 • Question: • The school provides information on agencies that can assist my child in grade level transition and/or transitions to post-schoolsettings. • In oneLEA: • 32.8% of the respondents disagreed with this at somelevel • District response: After analyzing their data, an LEA planned a Transition Fair to address thisneed 2019 OSEP LEADERSHIPCONFERENCE

42. Example2 • Question: • The school communicates regularly with me regarding my child’s progress on IEPgoals. • School A: 91.3% expressed some level of agreement with this statement • School B: 62.7% expressed some level of agreement with this statement • Where should valuable time and effort be spent? School B Be careful to not base decisions on smallresponses. 2019 OSEP LEADERSHIPCONFERENCE

43. Example3 • Question: • I am considered an equal partner with teachers and other professionals in planning and making decisions about my child’sprogram. • School C: 88.5% based on 26responses • School D: 0%, based on 1response • School E: 100.0% based on 2responses • District response: Address survey participation at schools D and E, share results with teachers and staff; non-participation is datatoo • Celebrate schoolC 2019 OSEP LEADERSHIPCONFERENCE

44. Example4 • Teachers ensure that I have fully understood the Procedural Safeguards [federal rules that protect the rights of parents] and my options if I disagree with a decision by theschool. • School A: 88% with some level ofagreement • School B: 68% with some level ofagreement • At the IEP meeting, we discussed accommodations and modifications that my child wouldneed. • School A: 92% with some level ofagreement • School B: 66%withsome level ofagreement • District response: provide professional learning to teachers/staff at School B regarding provision of Procedural Safeguards and how to conduct an IEP meeting 2019 OSEP LEADERSHIPCONFERENCE

45. Discussion: Making the Most of YourData • What ideas do you have for new ways to combine parent and family involvement data with other existingdata? • How have you seen states use parent and family involvement data to make improvements to programs and services for children with disabilities and theirfamilies? • What ideas do you have for new ways that states can use parent and family involvement data to informaction? 2019 OSEP LEADERSHIPCONFERENCE

46. IDC Resources and Opportunities forTA • Parent Involvement Toolkit –Making the Most of Parent Involvement Data: Improving Quality and EnhancingUnderstanding • Part C IDEA Data Process ToolkitandPart B IDEA Data Processes Toolkit • Peer-to-Peer Exchanges for Indicators C4 andB8 2019 OSEP LEADERSHIPCONFERENCE

47. ContactUs Amy Bitterman AmyBitterman@Westat.com Hadley Moore HadleyMoore@Westat.com Carol Seaycseay@doe.k12.ga.us 2019 OSEP LEADERSHIPCONFERENCE

48. For MoreInformation Visit the IDCwebsite http://ideadata.org/ Follow us onTwitter https://twitter.com/ideadatacenter Follow us onLinkedIn http://www.linkedin.com/company/idea-data-center 2019 OSEP LEADERSHIPCONFERENCE

49. 2019 OSEPLeadership Conference DISCLAIMER: The contents of this presentation were developed by the presenters for the 2019 OSEP Leadership Conference. However, these contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the FederalGovernment. (Authority: 20 U.S.C. 1221e-3 and3474) 49