Trend Data Guest Speakers: Pattie Johnson, TRI Sally Simich, ODE

1. Trend DataGuest Speakers: Pattie Johnson, TRI Sally Simich, ODE

2. Why Use Trend Data • Visual representation of multiple years of data to reveal a pattern of gradual change over time • To convey data clearly and accurately • To reveal or see patterns in the data • To motivate an audience to use data • To facilitate accurate and efficient interpretations of the data while minimizing the likelihood of misrepresenting data • Many individuals learn better through pictures than words; graphs help people remember the information

3. Graphs are Common • Inaccurate scale or axis • Wrong type of graph • Too much or too little data being displayed Bad Graph Contestants Forbes.com 2012

4. Considerations for Displaying Data • Scale should be to 100 when showing percents • Avoid 3D graphs • Consider the audience • More sophisticated audience = more sophisticated graph • Consider the task • What do you want the audience to do with the information • Know a specific percentage or number • (e.g., 79% of youth with disabilities are engaged one year after leaving high school) • Make comparisons • (e.g., males are more engaged than females) • See trends and patterns over time • (e.g., since 2010 the percent of youth engaged has steadily increased).

5. Questions Guiding a Trend Analysis • How representative are these data? • What direction are our outcomes going? • Are there differences in outcomes by subgroups? • Gender, Disability, Method of Exit, and Ethnicity • What is contributing to our outcomes? • How can we use the information?

6. Let’s look at some overall data trends across the four years of consistent data collection

7. All districts are required to participate each year. Student selection is based on a stratified sample designed to generate a representative sample of leavers to ensure generalizable information. • Districts select who will conduct the interviews, and data entry is open from June through September. Interviews can be completed with young adult or their family members. • Oregon’s PSO data have consistently been representative on all subgroups except dropouts PSO Data Collection

8. Sample Leavers, Response Rate, and Completed Interviews for Four Years Response Rate

10. What do we see in the trends? • Higher Education initial increase, then static • Competitive Employment Increasing • More Oregon leavers employed than in education • Other School and Other Work relatively unchanged • Not Engaged rate decreasing – right direction – until last year • Why look further? • To target resources to specific needs

11. Are there differences in outcomes by subgroups? • Gender x 3 years • Disability categories x 3 years • Ethnicity categories x 3 years • Method of Exit x 3 years

13. How representative are these data? • Explore the response size and how the subgroups matched the population • What direction are the outcomes going? • Look at graphs showing performance, trends, and comparisons • Examine outcomes by subgroups. • Work from general overview to more specific components • Explore what is contributing to the outcomes. • Look at a combination of components; extra questions asked as part of the survey; collect or examine other data Looking at Data: Process summary

14. District level summary reports • Engagement Report includes information on who is engaged by subgroup and if the sample of students interviewed matches the total district leavers • Interview Summary Report summarize your interview data, including the open ended responses given by the interviewee. Two Follow Up Report Formats

15. Monitor your progress on the 2014 data collection – constantly updated • Compare current district response rate, engagement rate, and representativeness to final results from 2012, 2011, and 2010 • Examine results of changes the district is making in transition services over time Using the Reports

16. . Goal 2013 Engagement Report

17. Sample District – subgroup

18. Sample District Data– Representative?

19. Compare results with like sized districts District and State comparisons

20. Questions for the district to consider when looking at reports • What do the most recent data show? • Can we make district-wide statements using our results, or have we missed dropouts or other groups? • Are our students going on to school within a year? • Are they able to get and keep jobs for at least 3 months? • What percentage of our students are actively working or learning within a year of leaving? • Do we have more than a third of our leavers who haven’t done any work or training after leaving high school? • Are we getting better at launching successful students?

21. How is ODE holding districts accountable for student PSO? • PSO is separate application available to secondary transition staff at district level. • Needed administrative audience • Now PSO will be included with monitoring indicators with notification if district failed to meet state targets

22. SPR&I Monitoring Report on Secondary Transition Indicators: PSO

23. SPR&I Monitoring Report on Secondary Transition Indicators: PSO

24. Accountability • What Is working for the district? • Where is more effort needed? • Administrators will need to explore the reports on the Post School Outcomes Application to learn about subgroup results and trends: • Disability • Method of exit • Gender • Race/Ethnicity • Like-sized districts

25. Next Steps for Districts • Share and discuss data with stakeholders • Identify areas of strength and areas for improvement • Use data to determine what changes are needed

26. For more information: Sally Simich Transition Specialist, Oregon Department of Education Sally.simich@state.ou.us 503-947-5639 Pattie Johnson Teaching Research Institute, Western Oregon University johnsop@wou.edu 503-838-8779 Charlotte Y. Alverson National Post School Outcomes Center, University of Oregon calverso@uoregon.edu 541-346-1390