Post School Outcomes: What Can We Learn from Trend Data?

1. Post School Outcomes: What Can We Learn from Trend Data? Pattie Johnson, WOU Charlotte Y. Alverson, UO Building Capacity Institute, 2013

2. Session Description • Oregon has three years of PSO data with consistent definitions for educational and employment outcomes. • What can we learn from examining the trends? • Where are the key areas that districts can use to evaluate progress toward increased engagement for their students? • Where will program changes have impact on improving performance?

3. Consistency • Since FFY 2008, states have had the same measure and definitions for Indicator 14, post-school outcomes. • With FFY 2011 data collection, we have 3 to 4 years of PSO data. • We can now start to examine trends in outcomes across years.

4. Data Collected Yearly by States States are measured on their implementation of IDEA through 20 Part B Indicators. #14: Percent of youth who are no longer in secondary school, had IEPs in effect at the time they left school, and were: • Enrolled in “higher education” • In “competitive employment” • Enrolled in “other postsecondary education or training” • In “some other employment”

5. Outcomes for Student with Disabilities as Measured by Indicator 14 United States Department of Education, Office of Special Education Programs (2011, 2012). Part B State Performance Plan/ Annual Performance Reports 2011 & 2012 Indicator Analyses.

6. Indicator 14 for Federal Reporting 1 HE 2 CE 3 OEd 4 OW 5 NE We will look at the five outcome categories because these are more meaningful for understanding our data and being able to use our data for program improvements

7. Questions Guiding the Analysis • How representative are these data? • What direction are our outcomes going? • Are there differences in outcomes by subgroups? • Gender: Male, Female • Disability: ID, ED, SLD, all other • Method of Exit: Regular diploma, Completed, Dropout • Ethnicity: Minority, Caucasian • What is contributing to our outcomes? • How can we use the information?

8. Looking at Data • How representative are these data? • Aggregate of response representativeness • What direction are our outcomes going? • Graphs of: • Overall A, B, & C Measure x 3 years • Overall 1, 2, 3, 4, & 5 x 3 years • Are there differences in outcomes by subgroups? • Gender Disability Method of Exit and Ethnicity categories x 3 years • What is contributing to our outcomes? • What supplemental survey questions will help answer this question?

9. PSO in Oregon • 1-year prior to conducting the survey, districts can collect accurate contact information on exiting students • All Local Education Agencies (LEAs) collect follow up data, larger LEAs are provided with prioritized list of leavers selected to achieve a representative sample of leavers based on race, disability, gender, and method of exit • LEA personnel conduct phone interviews • Responses are recorded in online secure website

10. How representative are these data?

11. Representativeness: Basic Numbers from Three Years • Oregon uses a stratified sample: • All districts conduct interviews with students each year • Small districts (15 leavers or less) interview all leavers • Larger districts are provided with a sample of required students to interview.

12. NPSO Calculator Representativeness: Combining three years of data Dropouts are under underrepresented - a finding consistent with the each years’ separate response analysis. Importance: to ensure sampled group represents state population, the difference should be 3% or less. Caution should be used in interpreting any results using the dropout category.

13. What direction are our outcomes going? • Overall A, B, & C Measures • Overall 1, 2, 3, 4, & 5 categories

14. National and OR State PSO Data Data Source: National aggregate of FFY 2009 SPP Submitted February 1, 2011; State data reported in the SPP FFY 2009 & APR FFY 2010, 2011

15. What direction are our outcomes going: Outcomes by Three Years

16. What do we see in the trends? • Higher Education initial increase, then static • Competitive Employment Increasing • More Oregon leavers employed than in education • Other School and Other Work relatively unchanged • Dip in middle year- 2010 • Slightly more leavers in Other Work than Other Education • Not Engaged rate decreasing – right direction • Why look further?

17. Are there differences in outcomes by subgroups? • Gender x 3 years • Disability categories x 3 years • Ethnicity categories x 3 years • Method of Exit x 3 years

18. Differences in Outcomes by Gender Percent Reported

19. Observations for Outcomes by Gender • More Females than Males in Hi Ed and both groups have fairly static trend • More Males than Females in Competitive Employment with increase in trend for Males • Other School engagement about the same rate for Females and Males • Other Employment similar rates, but Females have increasing trend over time • Not Engage decreasing trend for both groups

20. Differences in Outcomes by Disability Categories: SLD and ED

21. Differences in Outcomes by Disability Categories: All Other (Low Incidence) and ID

22. Observations for Outcomes by Disability • SLD: slight positive trend for HE and CE and slight negative trend for NE – trends going in desired direction • ED: negative trend in HE, but positive trend in CE; negative trend in NE- need to explore HE • AO/Low Incidence: Slight increase in CE, other engagement categories unchanged • ID: negative trend in HE, positive trend in all other engagement categories; highest group NE, but decreasing • Regardless of disability, about 1/3 of respondents are NE, HOWEVER, the trend is headed in the desired direction – seeing a negative trend in all disability categories - need to explore NE

23. Differences in Outcomes by Race/Ethnicity Categories

24. Combined Years for Sufficient Size of Race/Ethnicity Subgroups for Comparison

25. Observations for Outcomes by Race/Ethnicity • There are very minor outcome differences when all minorities are in one subgroup • There are insufficient interviews to look at trends over the 3 years by each minority group • Combining the data from the three years, as a representative sample was interviewed, allows further exploration • Differences are seen in the students combined into the minority subgroup that need to be explored further by the state and districts

26. Outcomes by Method of Exit category Regular Diploma

27. Outcomes by Method of Exit category Dropout

28. Observations for Outcomes by Method of Exit Groups: Regular Diploma and Dropout • Regular Diplomas: slight increases in HE and CE, and decrease in NE; trends are going in the desired directions • Dropouts – not representative of the state • Decrease in HE, slight increase in CE with a dip in 2010 • Slight increase in Other Education; static in Other Work • Decrease in NE

29. Summary Observations • Not Engaged – negative trend • Rate is slow • High number of youth in some subgroups • ID • Dropout • Higher Ed and Competitive Employment shifting to Other categories • ED – negative trend on HE with increase in Other Education

30. What is contributing to our outcomes? Supplemental survey questions can help answer this question.

31. Additional questions included on the follow- up interview in Oregon: • Do you have a drivers license? • What is your living situation? • What one thing would you tell your school? • Which independent activities can you do? • Do you receive benefits like co-workers? • What do you do for recreation? • If you haven’t worked, why not? • Have you received support from adult Agencies?

32. What Agency Services have you accessed since leaving school? The list of agencies on the follow-up interview includes: • Social Security Disability Insurance or Supplemental Security Income • Developmental Disability services • Office of Vocational Rehabilitation • Temporary Assistance for Needy Families • Supplemental Nutrition Assistance Program • College Disability Services • Loans, Financial Aid

33. Change in Outcome Classification • For the next series of charts, the outcome groups were modified to allow a closer look at students who tried school or work, but were not successful. • 1 HE 2 CE 3/4 Other ed/work 5-Attempted 5-None • 5-Attempted: students answered Yes to either school/ training or employment, but did not continue long enough to qualify as ‘engaged’ • 5-None: students reported NO school/training or employment experience

34. Differences in Outcomes by Agency Services Received

35. Differences in Outcomes by Agency Services Received

36. How Can We Use These Data: Next Steps • Finalize the analysis • Other supplemental questions • Share and discuss trend data • ODE Transition Specialist • Transition Advisory Council Stakeholders • Agency Partners: ODDS and VR • Determine what data to share and how to share • District and School Stakeholders

37. Looking at Data: Process summary • How representative are these data? • We explored the response size and how the subgroups matched the population • What direction are our outcomes going? • Looked at graphs showing performance, trends, and comparisons • Are there differences in outcomes by subgroups? • Worked from general overview to more specific components • What is contributing to our outcomes? • Looked at a combination of components, modified the question if necessary, and summarized what we learned at each step

38. For more information: • Pattie Johnson • Teaching Research Institute, Western Oregon University • johnsop@wou.edu 503-838-8779 • Charlotte Y. Alverson • National Post School Outcomes Center, University of Oregon • calverso@uoregon.edu 541-346-1390