Understanding and Using Data Effectively

1. Understanding and Using Data Effectively By The DATA unit Data, Accountability, and Technical Assessments OCTE January 18, 2011

2. Presenters • David MacQuarrie • Krishnan Sudharsan 1

3. Overview • Benefits of Participating in a Data Analysis Process • Consolidated Annual Report (CAR) • Secondary Core Performance Indicators (CPIs) and Regional GIS maps • Data Analysis Example (State CPIs) • Data Analysis Activity (Region/CEPDs) • Wrap Up 2

4. Benefits of Participating Your CEPD team will: • Make informed decisions that will result in an increase in student achievement • Use a process to analyze data for program improvement 3

5. Consolidated Annual Report • Reports to OVAE on Michigan’s CPIs • 8 secondary and 6 Tech Prep CPIs • Performance of 25 Regions and 25 Tech Prep Consortia • Secondary, Post Secondary, and Tech Prep • Improvement plans (when 90% target not met)

6. Secondary CPIs • 1S1: Academic Attainment in Reading • 1S2: Academic Attainment in Mathematics • 2S1: Technical Skill Attainment • 3S1: School Completion • 4S1: Student Graduation Rate • 5S1: Placement • 6S1: Nontraditional Participation • 6S2: Nontraditional Completion

7. New Ethnicity Categories

8. CTEIS Data

9. CTEIS Data

10. CTEIS Data • Students with Disabilities: • 11% ± .5% over the past four years • Non-Traditional Participation: • 9.03%, increase of 1% in four years

11. Secondary CPIs: CAR

12. Secondary CPIs: CAR • 10 out of the 25 regions met all 8 indicator targets • 15 regions failed to make 90% of at least one indicator • The number of regions not meeting 90% of the Adjusted Level of Performance (ALP) improved for indicators 1S1 (from 5 to 1 regions), 1S2 (from 6 to 1 regions), and 6S2 (from 25 to 4 regions) compared to last year

13. Secondary CPIs: CAR • The number of regions not meeting 90% of the ALP worsened for indicators 2S1 (from 0 to 1 regions) and 6S1 (from 8 to 12 regions) compared to last year • All 25 regions exceeded the ALP for Indicators 3S1, 4S1, and 5S1 in 2009-2010 and 2010-2011

14. Secondary CPIs 2012 Targets

15. 1S1: Academic Attainment Reading • Numerator = Number of CTE concentrators who have met the proficient or advanced level on the statewide high school reading/language arts assessment • Denominator = Number of CTE concentrators who took the ESEA assessment in reading/language arts and who, in the reporting year, left secondary education

16. 1S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (47.50%)

17. 1S2: Academic Attainment in Mathematics • Numerator = Number of CTE concentrators who have met the proficient or advanced level on the statewide high school mathematics assessment • Denominator = Number of CTE concentrators who took the ESEA assessment in mathematics and who, in the reporting year, left secondary education

18. 1S2: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (45.00%) 1S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (47.50%)

19. 2S1: Technical Skill Attainment • Numerator = Number of CTE concentrators who passed technical skill assessments that are aligned with industry-recognized standards, if available and appropriate during the reporting year • Denominator = Number of CTE concentrators who took assessments during the reporting year

20. 2S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (35.00%)

21. 3S1: School Completion • Numerator = Number of CTE concentrators who earned a regular secondary school diploma, earned a General Education Development (GED) credential or earned a state-recognized equivalent, during the reporting year • Denominator = Number of CTE concentrators who, in the reporting year, were included in the state’s computation of its five-year graduation rate

22. 3S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (90.00%)

23. 4S1: Student Graduation Rates • Numerator = Number of CTE program concentrators who, in the reporting year, were included as graduated in the state’s computation of its graduation rate • Denominator = Number of CTE program concentrators who, in the reporting year, were included in the state’s computation of its graduation rate as defined in the state’s Consolidated Accountability Plan pursuant to Section 1111(b)(2)(C)(vi) of the ESEA

24. 4S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (80.00%)

25. 5S1: Placement • Numerator = Number of 11th and 12th grade CTE completers who left secondary education and were placed in postsecondary education or advanced training, in the military service, or employment in the third quarter following the program year in which they left secondary education • Denominator = The number of 11th and 12th grade grade CTE completers who left secondary education during the reporting year

26. 5S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (94.79%)

27. 6S1: Nontraditional Participation • Numerator = Number of CTE participants from underrepresented gender groups who participated in a program that leads to employment in nontraditional fields during the reporting year • Denominator = Number of CTE participants who participated in a program that leads to employment in nontraditional fields during the reporting year

28. 6S1: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (22.50%)

29. 6S2: Nontraditional Completion • Numerator = Number of CTE participants from underrepresented gender groups, who completed a program that leads to employment in nontraditional fields prior to leaving secondary education and who left school in the reporting year • Denominator = Number of CTE participants from underrepresented gender groups, who had participated in a program that leads to employment in nontraditional fields and who left school in the reporting year

30. 6S2: Regions that meet, do not meet, and those within 90 percent of the 2010 State Target (66.56%)

31. Thought Question/Exercise • Cut off scores for the MME increased beginning Fall 2011 • Question 1: How many think this will lead to an increase in 1S1 and 1S2? • Question 2: What percentage change do you expect to see in 1S1 and 1S2 (Plot)?

32. Why is Data Important? • 200,000 (Data) • “Your bank balance has jumped by 1000% to $200,000” (Information) • “Nobody owes me that much money” (Knowledge) • “I'd better talk to the bank before I spend it, because of what has happened to other people” is wisdom

33. Process to Analyze Data • Gather the data and input into table • Graph/plot the data • Analyze and describe patterns in the data • Develop hypotheses • Report out a pattern, hypothesis, & intervention • Science starts with developing hypotheses and then testing them

34. Hands on Activity • Excel sheet with regional and CEPD CPIs • MS Word form for region • Instructions for using CTEIS report generator • Instructions for plotting a trend line in Excel • PDFs of CPIs by Race & Gender for regions and CEPDs

35. 1S1: Statewide Trends

36. 1S1: Regional Trends • How has your region performed on 1S1 in the last 4 years? • What is the linear rate of change over the 4 years? • How does your regional performance compare to the state? • Are there performance differences across gender in your region? Why? What can you do to minimize these differences?

37. CPI Trends across CEPDs • Examine the graphs for the CEPD/s within your region. • How are the CEPDs doing compared to each other? How is their performance affecting the regional CPIs? • What strategies do you have for sharing successful strategies across CEPDs?

38. 6S1: Statewide Trends

39. 6S1: Regional Trends • How has your region performed on 6S1 in the last 4 years? What is the linear rate of change over the 4 years? • What strategies have you implemented to improve 6S1? What strategies have worked and not worked? • Do you have any other thoughts/ideas for improving nontraditional participation rates?

40. Report Out • Report on a pattern that you observed in your data (1S1 or 6S1)? • What hypothesis did you develop to explain this pattern? • What intervention would help alleviate the issue?

41. 6S2: Statewide Trends

42. 6S2: Regional Trends • How has your region performed on 6S2 in the last 4 years? What is the linear rate of change over the 4 years? • What strategies have you implemented to improve 6S2? What strategies have worked and not worked? • Do you have any other thoughts/ideas for improving nontraditional completion rates?

43. 1S2: Statewide Trends

44. 1S2: Regional Trends • How has your region performed on 1S2 in the last 4 years? • What is the linear rate of change over the 4 years? • How does your regional performance compare to the state? • Are there performance differences across race in your region? Why? What can you do to minimize these differences?

45. 2S1: Statewide Trends

46. 2S1: Regional Trends • How has your region performed on 2S1 in the last 3 years? • What is the linear rate of change over the 3 years? • How does your regional performance compare to the state? • Are there performance differences across disability status in your region? Why? What can you do to minimize these differences?

47. 3S1: Statewide Trends

48. 3S1: Regional Trends • How has your region performed on 3S1 in the last 4 years? • What is the linear rate of change over the 4 years? • How does your regional performance compare to the state? • Are there differences across programs in your region? Why? What can you do to minimize these differences?

49. 4S1: Statewide Trends

50. 4S1: Regional Trends • How has your region performed on 4S1 in the last 4 years? What is the linear rate of change over the 4 years? • How does your regional performance compare to the state? • What is the graduation rate for general education students in your region? How do CTE students compare with general ed.? Why?