How’s it Growing? Enhancing Data Literacy and Engagement through Data Visualization

1. How’s it Growing?Enhancing Data Literacy and Engagement through Data Visualization Juan D’Brot Executive Director Office of Assessment and Accountability October 22, 2012

2. The Overview • What is Growth and a Growth Model? • Growth at the school level • Applying growth (FAIPs) • Drilling back down to the student level • Applying growth in the classroom • Where to go next to drive instructional decisions

3. Why a Growth Model? • AYP vs. Continuous Improvement (non- accountability) • Enhancing Data Literacy • Enhancing Engagement • Future-oriented Capacity Building • SMARTER Balanced Assessment Consortium • ESEA Reauthorization

4. Why NCLB Promotes Insensitivity to Individual Student Growth and School Improvement No Content Mastery No Content Mastery Full Content Mastery Full Content Mastery AYP (status) Model Student Proficiency Growth (improvement) Model Access to Full Spectrum of Student Performance and Improvement

5. Compliance vs. Continuous Improvement Adequate Yearly Progress Improvement and Growth • Snapshot • Non-comparable trends • Funding considerations • “Outcome” • Multiple Snapshots • Comparable trends • (More) instructional consideration • “Process”

6. Transitioning to Growth • Past: Contextualizing prior performance • Present: Understanding current student and school growth • Future: Building capacity for understanding and applying growth post-WESTEST 2

7. The WV Growth Model • What are we trying to answer? • Purpose • How it works • What it yields? • How we interpret it • What levels (overview) • Past, present, future • Student, Grade, School, District, State • Prior growth, observed growth, projected growth (Growth to Standard)

8. Asking the Right Questions… • Until you’ve defined the question, you cannot examine the appropriate data. • Why? • Different data answer different questions • Different questions lead to different conversations

9. Fact: …increased use of student assessment data as a “significant factor” in (insert favorite accountability topic here…) • But what student data? That leads us to start asking questions… • But what questions? • Whose questions?

10. Whose Questions? WHY? To answer questions about the student How? By not declaring a verdict (status). Instead, by changing the conversation (to growth and progress)… • Researchers • Administrators • Teachers • Parents • Policymakers • Business Leaders • Community Members • Guardians

11. Fact: Assessment Systems try to be Everything to Everyone • Danger: answering too many questions with the same data • How to Differentiate Instruction • Determining Student Proficiency • Measuring Class Performance • Determining AYP • Possible Solution: Can we come up with a common question across stakeholders? But first…

12. The WV Growth Model: Our Purpose • To provide an answer to a few common questions: • “How much did my student grow this year?” • “What does this growth mean compared to everyone else?” • “Is it enough growth?” What do these three questions have in common? Information about students

13. Status vs. Growth • Before growth, let’s review terms • Status: A snapshot measure of a single point in time – “Proficiency” • Growth: Multiple snapshots across many points in time

14. Status vs. Growth A Balanced Approach

15. Let’s Consider Growth at a High Level…

16. Measuring Growth: A Primer • What is the likelihood of current (and future) performance given past performance? • Based on students who have similar academic histories (academic cohort) • Uses all available data to answer these three questions: • Am I proficient? (Status) • How much did I grow? (SGPs) • Is it enough growth? (Growth to Standard)

17. Our Growth Options (Primer, cont’d) • How much growth? – Starts with the student. Can be aggregated to drive school conversations • High Growth (a good surprise) • Typical Growth (a statistical expectation) • Low Growth (a bad surprise) • Is it enough growth? – Can drive studentlearning conversations • Catching Up • Keeping Up • Falling Behind

18. Questions - Revisited • “How much did my student grow this year?” (Time 2 – Time 1) – We can see this today • “What does this growth mean compared to everyone else?” (Normative Component) • “Is it enough growth?” (Criterion Component - CSOs)

19. For more information… Please visit the WVDE’s Growth Model webpage: http://wvde.state.wv.us/growth/

21. Understanding Grade Level and School Growth Reports

22. Viewing Aggregated Growth at the School Level • What to consider? • High vs. Low Achievement • Individual Student Performance • Overall Percent Proficient • High vs. Low Growth • Individual Student Growth • Overall Growth for a School • An example of a grade and school level report

24. Grade Level Bubble Chart • Growth Percentile vs. Scale Score (Achievement) for all students within a grade that have growth data • Shows distribution of a group of students • Hover capability to view student information at each data point (bubble)

25. Above Mastery Mastery Partial Mastery Novice

26. Why are we interested in these data? Easy to read overview of grade level performance Growth + performance is more telling than performance alone Ease of access can lead to immediate interest into student performance Helps support our next logical question… Higher Performance of a Single Student John Smith:2010-11 student Growth Percentile (Target): 95 (14) 2010-11 Achievement Level (Scale Score): Distinguished (757) Distinguished Each bubble is a single student who is plotted by: Scale Score Above Mastery Mastery Partial Mastery Individual Student Growth Percentile Novice Higher Observed Growth of a Single Student in a School

27. School Growth and Achievement Bubble Plots

28. School Growth and Achievement Bubble Plots

29. Enhancing Interest in the Story Behind the Data Growth Model  Interest in Data Examining Individual Performance  Reviewing Individual Item Analysis  Consideration of how to more effectively use of interim assessments and formative processes  Validating what we see students learning and understanding Teacher expertise leads to professional judgment about instruction

30. Examining District-LevelStudent Reports • Log in to WVEIS and go to the Private Data Site • At the bottom of the screen, you will see a link titled “Growth Model” • Click Growth Model, select your district and select a school of interest. • Click Go!

32. Growth Legend Content Areas Math (pdf)Reading (pdf) Click Here for Bubble Plots Math (pdf)Reading (pdf) Math (pdf)Reading (pdf)

33. Subgroup Comparison Growth Legend Content Areas

36. Understanding Individual Student WESTEST 2 Growth & Achievement Report

37. Achievement: A Quick Review • Current WESTEST 2 Student Report shows: • Single Scale Score in each content area • Level of Achievement (Performance Levels) • Distinguished • Above Mastery • Mastery • Partial Mastery • Novice

38. The WV Growth Model: Our Purpose • To provide answers to a few common questions: • “How much did the individual student grow this year?” • “What does this growth mean compared to everyone else?” • “Is it enough growth?” What do these three questions have in common? Information about students

39. Measuring Growth: A Primer • What is the likelihood of current (and future) performance given past performance? • Based on students who have similar academic histories (academic cohort) • Uses all available data to answer these three questions: • Am I proficient? (Status) • How much did I grow? (SGPs) • Is it enough growth? (Growth to Standard)

40. Our Growth Options (Primer, cont’d) • How much growth? – Starts with the student. Can be aggregated to drive school conversations • High Growth (a good surprise) • Typical Growth (a statistical expectation) • Low Growth (a bad surprise) • Is it enough growth? – Can drive studentlearning conversations • Catching Up • Keeping Up • Falling Behind

41. Meet the Student Growthand Achievement Report Hello. It’s nice to meet you. Hello, Growth Report

42. Name of Student Name of School Interpretation Guide Growth and Achievement for Math Growth and Achievement for R/LA Suggested Uses Partial Mastery Partial Mastery Mastery Novice

43. Scale Scores for Math Scale Scores for R/LA WESTEST 2 Achievement Levels WESTEST 2 Achievement Levels Grade Level and School Year Grade Level and School Year Legend for Math Student Achievement Levels Student Achievement Levels Legend for R/LA Partial Mastery Partial Mastery Mastery Novice

44. The colored arrows between the dots represent growth from one grade to the next. The color of the arrow signifies the level of growth. The fan represents how the student might score next year under several growth scenarios. High growth is indicated in green, typical growth in white, and low growth in red. Growth Percentiles Growth Levels Partial Mastery Partial Mastery Mastery Novice

45. Is it enough growth? How much growth?

46. How much growth? Growth Percentile – 1% to 99% • 66%-99% = High Growth = Green Arrow - Good surprise - the student exceeded what was expected • 35%-65% = Typical Growth = White Arrow - Statistical expectation - the student grew as expected • 1%-34% = Low Growth = Red Arrow - Bad surprise – The student grew less than was expected

47. Is it enough growth? Will a student be proficient next year? • Determined at only one grade level at a time • Depends on the achievement level where the student was at a particular grade level • The fan can help us understand potential stories for a given student • What level of growth is required for a student who is proficient to keep up with proficiency next year? • What level of growth is required for a student who is not proficient to catch up to proficiency next year? • What level of growth for any student, would cause a student to fall or stay below proficiency next year?

48. Prior Proficient: Student is Keeping Up

49. Prior Proficient: Student is Not Keeping Up/ Falling Behind (Math)

50. Prior Proficient: Student Keeping Up (Math)/ Will Fall Behind (R/LA)