Reporting Post-school Outcome Data May 2008

1. Reporting Post-school Outcome DataMay 2008 Cinda JohnsonMary Kampa Center for Change in Wisconsin Post High Survey/ Transition Services Indicator 14 Seattle University CESA #11 Seattle, WA Turtle Lake, Wisconsin cinda@seattleu.edumaryk@cesa11.k12.wi.us 206-296-5888 715-986-2020

2. History:Washington • Data collection beginning in 1980 with Edgar’s work • Consistent post-school outcome data collected since 1996, (Edgar, et al.) • Consensus data collection (all districts, all leavers) beginning in 1998 • 248 school districts, 5,000 leavers • 80% contact rate

3. Methods:Washington • Participants: Special education graduates and drop-outs from all 248 school districts. • Annually: Demographic information collected in year prior to leaving school. • Information collected from final IEP. • Telephone survey conducted with youth or family member within one year of graduating or dropping out (WA has one diploma).

4. Methods: Washington • Washington State Post-school Survey (http://www.seattleu.edu/ccts/post-school_survey.asp) • Secure password protected website

5. Instrumentation:Washington • Survey questions includes: • Demographics including gender, age, disability • Post-secondary goals from final IEP • Agency linkages on final IEP • Data gathered includes: • Post-school outcomes in post-secondary education, training and employment • Details of those outcomes (type of school, training program, job, wages, hours, etc.) • Agency linkages

6. Procedures:Washington • Training: http://www.seattleu.edu/ccts/training.asp • Data manager at state level • Data manager at district level • Data users at district level • Confidentiality, assure consent • IRB agreements

7. Procedures:Washington • Districts are responsible to conduct interviews with former students. • Teachers conduct the majority of interviews. • Training for interviewers is provided on-line, teleconferences, on site, meetings, trainings, and conferences. • At least 3 attempts made to reach youth during different times of day.

8. Table School Staff Completing Post-School Surveys 2006 Graduates Interviewed

9. Reporting Post-school Data:Washington • Reports to state and districts. • Data disaggregated by district, region, county, high school. • Outcomes compared to state and previous years. • Data reported to agencies (DVR, DDD), Governor’s office, ESDs and parent groups. • Assure confidentiality (cell size).

10. Reporting Data for Program Improvement:Washington • Leadership • Facilitator • Examine the data with colleagues • Participate in surveying former students and share the stories • Develop goals based on the data • Tie post-school outcome data to school improvement activities

12. Post-school Outcome Data: Washington • Collecting (by whom and when; from whom) • Analyzing (representative of pool) • Reporting (LEA’s, SEA, Public, APR, SPP) • Setting goals and benchmarks (SEA, LEA’s) • Informing practices • Improving outcomes

13. Using Post-school Outcome Data to Inform Change: LEA, Washington • Competitive employment for youth with developmental disabilities was low • Met with agencies to clarify language and definitions (competitive employment) • “Carved-out positions” were not leading to competitive employment • Assessed OJT’s with agencies and modified to meet criteria of skills defined for competitive employment

14. History:Wisconsin • Began in 2001 with 1999-2000 exiters • Consistent core questions : based on NLTS and other surveys at the time, including WA, TX and OR • Dual data collection method: Statewide sample(odd years) and individual district census(even years) • 400 school districts in state • Nearly 8,000 exiters with disability annually • app. 81 districts and Milwaukee (13 schools) each year • Census within selected districts • 70% response rate (districts = 80%); lower in 2007 (38%) due to change in method

15. Methods:Wisconsin • Each LEA in Wisconsin must participate in an outcomes survey once between 2006-07 and 2011-2012 to comply with SPP Indicator #14. • LEA participation is aligned with the DPI Self-Assessment Monitoring Cycle • Within one year of exiting, contact former students who exited with IEP and: • A regular diploma • A certificate of attendance • Reached maximum age of eligibility • Dropped-out

16. Methods:Wisconsin • The Wisconsin Post High School Outcomes Survey (WPHSOS) (www.posthighsurvey.org)is a web-based tool designed to ensure consistency in data collection and reporting requirements of SPP Indicator 14. • Outcomes information is collected from former students and their families through a telephone interview conducted on behalf of the former student’s high school by a professional survey center.

17. Methods:Wisconsin • District results can be viewed immediately and used for transition data analysis and planning • LEAs may also use the WPHSOS website to conduct their own surveys or include exiters without disabilities when not participating in the statewide survey

18. Procedures:Wisconsin • Work with school districts to prepare them for a two year data collection process • Year 1: Districts - • Collect contact information all year for youth in final year • Track drop-out contact information • Explain upcoming survey to youth and parent • Give questionnaire to youth and parent • view and use www.psocenter.org resources • Year 2: Districts - • Feb: Verify former student contact information • March: Send former students district and SEA letters • April-June: LEAs surveyed/view survey results

19. Procedures:Wisconsin • SEA Data manager prepares exiter information based on LEA Exiter Report • Data transferred through secure FTP site to outcomes website • Interviewers and district apply for user name and password; sign confidentiality agreement

20. Procedures:Wisconsin • 5 - 8 attempts made to reach each exiter; different times of day, weekends, special operators, other languages, jail, military • Responses also accepted from family member if knowledgeable about HS and current activities • Training for district interviewers is provided using developed training materials

21. Instrumentation:Wisconsin • Survey questions include: • Living situation, community participation, social activities, adult agencies and community supports • Postsecondary education and training – types, accommodation and disclosure • Employment – type, setting, hours, wages, benefits, accommodations • HS experiences/IEP plans • Open-ended questions – if not living, working or going on to school, why not? • What is something positive that happened in HS that helped you met your goals?

22. Post-school Data:Wisconsin • LEAs access outcomes data at district and building level as it is collected • Statewide data available Sept. 1 • All survey questions disaggregated by gender, ethnicity, disability, exit type, HS • Can create additional reports by region, county, CESA, school size, other as requested • Cell size for confidentiality is 5

23. Post-school Data:Wisconsin • Only Indicator 14 “%” is publicly reported • LEA is provided: • District GEDE Report • District Summary Report • District Report Starter • Improvement Planning Tools • Youth Leadership Council will provide youth prospective and take an active role in co-presenting with WPHSOS director at regional trainings and conferences

24. Post-school Data:Wisconsin • Districts collect outcomes data because they have to • Districts use outcomes data because they • have to • want to

25. Post-school Data:Wisconsin • HAVE to: • Focused-Monitoring– identified by DPI as in need of focused-monitoring; districts are required to participate in improvement planning strategies (e.g. Indicator 1 - Graduation Rate) • Indicator 14 Districts– identified by DPI as “in need of assistance” e.g. districts 30% or more below State Indicator 14 • WANT to: • Districts seeking assistance with data analysis – statewide data tools can be easily used in districts or CESAs involved in data analysis, improvement planning or retreats • WSTI TAT Districts – Participate in the development of a pilot professional development that includes data analysis for the Year 2 report-out (mini-data retreat - LEAs use their data to identify weaknesses and improvement strategies)

26. Goals of the WPHSOS:Wisconsin • The WPHSOS will: • Align identified goals and activities of the post school follow-up project with WSTI (Wisconsin Statewide Transition Initiative) and SPDG (State Professional Development Grant), using a complementary technical assistance approach among projects to help districts use data to improve outcomes • Provide technical assistance to move the outcomes website from a data collection and reporting tool to a tool LEAs use to identify local needs and determine where improvement strategies are needed to positively impact Stakeholder Advisory identified Indicator 14 SPP Targets • Develop statewide information sharing,resources and sustained professional development on Indicator 14

27. Data Sharing:Wisconsin • Several important pieces in place at the current time: • Indicator 1 and 2 data are shared with the post high website, and are connected with Indicator 14 outcomes reporting • Indictor 13 data and Indictor 14 data share a common web designer • Databases are connected currently create a joint report: Indicator 1, 2, and 14 data are imported into the “Transition in IEP Checklist Report” • Individual student identification numbers (though not yet shared with the websites)

28. New Direction:Wisconsin Taking the website from data collection to data usehelping districts go from “why do we need to collect this information?” to “how can we use our results to effect change?” State & Compliance (expert) District (expert) Administration Teachers YouthParentsCommunity

29. New Direction:Wisconsin Dissemination • Shift from working with just directors of special education (because it is password protected information and “special education”) to information dissemination to all district administration: • State School Board Convention • State Superintendent’s Conference • State Secondary Principal’s Conference • State School Psychology Convention

30. Sharing Post-school Data:Wisconsin • Conferences/Poster Sessions/General Information • National, State/Regional, Local • State Superintendent’s Leadership Conference • WI. Council of Administrators of Student Services • Wisconsin Rehabilitation & Transition Conference • Wisconsin Statewide Transition Conference • DPI presentations and Stakeholder Advisory Meeting • CESA board/RSN meetings upon request • Department of Workforce Development • Department of Health and Human Services • State Sheltered Workshop Group

31. Sharing Post-school Data:Wisconsin • Conferences/Poster Sessions/General Information • TAC, TAN and TAT meetings (www.wsti.org) upon request (e.g. Washburn Co. - transportation grant and Barron Co. – county council activities) • Others upon request and availability • Community of Practice Groups/ Work Groups • Wisconsin Community on Transition and Practice Groups (www.sharedwork.org) • SPDG – statewide transition project, data, newsletters, conference planning, needs assessment • Print Materials/Resources • Resources/Professional Development • Statewide Outcomes/Indicator 14 Reports/Newsletters

32. Wisconsin Indicator 14Any Postsecondary Ed./Training by Survey Year

33. Wisconsin Indicator 14Postsecondary Education Summary • Overall participation in postsecondary education and training has been consistent over time (46% - 48%) • Types of postsecondary education have changed • a higher % in 2-Yr, 4-Yr and Tech College • a higher % of female exiters participating • significant increase in participation in Tech College • participation in 4-Yr increasing more than 2-Yr • Inclusion of dropouts did not affected the overall % of participation(46% vs 47%),although as a group, dropouts participate less in all types of postsecondary ed./training

34. Wisconsin Indicator 14Any Employment by Survey Year

35. Wisconsin Indicator 14Employment Summary • Employment from 1999-01 to 2003-04 decreased but rebounded in 2005-06 • Employment over time: • Employment in the community has increased • Hours per week worked have remained stable • Wages have increases slightly • Beginning with 2005-06, “competitive employment” to be included in employment, for Indicator 14

36. Wisconsin Indicator 14Employment Summary • Several HS indicators have changed • Fewer report paid employment while in HS • Fewer are obtaining a valid driver’s license in HS • Inclusion of drop-outs did not affected the overall % of those competitively employed (33% vs. 34%) • As a group, dropouts are employed at comparable rates, but a higher % • work more hours per week • earn less per hour

37. WisconsinIndicator 141999-2000 to 2005-06 Major Exiter Outcomes

38. Wisconsin Indicator 14Summary of Major Outcomes 79% = 283 of youth who had IEPs and are no longer in secondary school have been competitively employed (full-time or part-time), enrolled in some type of postsecondary school, or both, within one year of leaving high school divided by the 358 youth assessed who had IEPs and are no longer in secondary school. 283/358 = 79% 2007 Report of 2005-06 Indicator 14 Baseline Data (n = 358) t

39. Wisconsin Indicator 14Summary of Major Outcomes • Data collection on youth with disabilities one year after exiting high school indicates: • There is a consistent % of youth attending postsecondary education or training over time • The % of youth employed increased last year, following three years of decline in employment • High school factors may influence competitive employment post high school • Fewer than ½ of youth who are employed are “competitively employed”

40. IDEA Partnership Grant:Wisconsin • Review the available evidence-based practices and research on NSTTAC, NDPC-SD, NPSO, and other sources, addressing both rural and urban concerns • Begin creating a database of frameworks, strategies and factors that are connected to post high outcomes • Begin creating a tool LEAs and teachers can easily access and use in data analysis and planning of professional development activities

41. IDEA Partnership Grant:Wisconsin • Synthesize this information and identify: • District/building level: e.g. curriculums, engagement and school climate surveys, how and what HSs implement in their buildings (may be related to HS Re-design), connections with Indicators 1 and 2 • Teacher/classroom: material and activity specific: e.g. programs, curriculums, activities • Youth /parents/family: related to known risk or success factors: e.g. HS employment, not failing more than two classes, attendance, reading scores, poverty factors, information from Indicator 13, state or district assessment scores, IEP goals, specific transition or outside agency services, senior exit survey • Community/Adult Service Providers: DHFS, DVR, employment agencies, independent living centers, etc.

42. Final Thoughts - Washington • Post-school outcomes may not increase at the aggregate level • More respondents (harder to reach included) • Dropouts • Economy • Attention to Indicator 14 at disaggregate level may improve Indicator 13

43. Final Thoughts - Wisconsin • COLLECT: Beyond the Indicator 14 percentage, get outcomes data to the LEA and teacher level • USE: Once district is looking at the data, have improvement resources readily available • SHARE: with statewide and regional partners – help them access and use data • IMPROVE: Local improvement will lead to increased state Indicator 14

44. Contact Information • Cinda Johnson, Ed.D. Seattle University 206-296-5888 cinda@seattleu.edu Center for Change in Transition Services www.seattleu.edu/ccts

45. Contact Information • Mary Kampa, CESA #11 715-986-2020 maryk@cesa11.k12.wi.us