Understanding Pennsylvania Value-Added Assessment System (PVAAS)

1. Pennsylvania Value-Added Assessment System (PVAAS):PVAAS Public Site Tour PVAAS Statewide Team for PDE Fall 2011

2. Session Description This 60 minute session will provide an overview of the PVAAS public reporting site. All data reported on the public site will be described as to the intent and meaning of the results. This includes the district and school level data, the School Search feature and the web-based scatter plots. Materials and resources to use as tools for communication about PVAAS will be reviewed. Opportunities for questions and discussion will be provided.

3. Agenda Achievement and Progress Data What’s New? Available Data & Features Website Demonstration Suggested Resources for Communicating PVAAS Questions

4. Achievement vs. Progress Achievement Progress Dependent upon what happens as a result of schooling Not correlated with demographic factors Is the concept underlying value-added analysis and reporting • The final result of an academic experience • Highly correlated with demographic factors, such as socioeconomic status • Affected by factors outside the school

5. Achievement vs. Progress Achievement Progress Measures students’ progress across time/years Compares student performance to his/her own prior performance Critical to ensuring students’ future and continued academic success • Measures students’ performance at a single point in time • Compares student performance to a standard • Critical to students’ post-secondary opportunities

6. Stair-Step Expectations Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test In a perfect world students would start school at about the same academic level. Proficiency levels could simply be achieved by students gaining age appropriate knowledge and skills each year.

7. Differentiated Reality Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test Students begin school at different places, and they often progress at different rates. Yet all students are expected to achieve the same level of academic proficiency each year.

8. The Need for Progress Measures Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test To measure school effectiveness in this “differentiated” world, we need to pay attention not only to proficiency levels but also to how much progress students make in a given year.

9. What is Value-Added? • Imagine a child’s physical growth curve • Parent or doctor measures the child’s height at various ages • A graph can be constructed to illustrate the height of the child as shown here

10. What is Value-Added? • Let’s apply the same process to education • This chart measures the growth, or progress, for a group of students at various grade levels • This is value-added  following the growth, or progress, of students over time to estimate growth during a year of schooling. With PVAAS, Pennsylvania educators get an indicator as to whether they are making appropriate academic progress for their students!

11. Value-Added is… • A statistical analysis used to measure a district’s or school’s impact on the academic progress rates of groups of students from year to year. • Conceptually, a growth measure is approximately the difference between current achievement (current results) and prior achievement (prior results) with achievement being measured by an appropriate assessment, such as the PSSA. • PVAAS is NOT a simple comparison of two scores!

12. Value-Added & PVAAS • Pennsylvania’s model for value-added is called PVAAS - the Pennsylvania Value-Added Assessment System. • PVAAS is based on the EVAAS Methodology – the Education Value-Added Assessment System. • The EVAAS methodology has been nationally reviewed and published.

13. Benefits of Value-Added • Offers an objective, more accurate way to measure the influence districts and schools have on students’ academic progress: • Administrators and Teachers can: • Monitor the progress of all groups of students from low-achieving to high-achieving ensuring growth opportunities for all students • Make more informed, data-driven decisions about where to focus resources to help students make greater progress and perform at higher levels. • Align professional development efforts in the areas of greatest need. • Identify best practices and implement instructional strategies and programs that best meet the needs of all students.

14. Key Concepts in Understanding PVAAS Reporting • PVAAS reporting reflects the effectiveness of your district’s or school’s Standards-Aligned System • PVAAS reporting reflects the district or school’s system regarding curriculum, assessment, and instruction

15. Overview of PVAAS Public Reports & Features

16. Achievement + Growth Achievement results (PSSA) and growth results (PVAAS) must be used together to get a complete picture of student learning. To view the achievement results for Pennsylvania's public districts/schools, go to: http://paayp.emetric.net/

17. PVAAS Login Page https://pvaas.sas.com

18. Public Home Screen

19. PVAAS Public Reports Use of Reports Tab to Select & View Reports

20. PVAAS Public Reports Use of Tests Tab to View Reports at Different Grade Levels for the District and School Value-Added Summaries

21. PVAAS Public Reports • Value-Added Summary Reports • District/LEA and School Level data only • Math and Reading • Grades 4-8 and 9-11 • Science and Writing, Grades 9-11 • School Search Capability • Allows users to find and view the progress of local schools, charter schools, and full-time CTCs across Pennsylvania. • Can search for similar schools based on grade levels tested, various demographics, Intermediate Unit (IU) region and/or county.

22. PVAAS Public Reports (continued) • Web-based scatter plots • Math, Reading • Grades 4-8 & Grades 9-11 • Science and Writing, Grades 9-11

23. Reports on Public Site The data on the PVAAS public site come directly from the reports on the district/school password-protected site. This session will include a cross-walk between these two sites to show the source of the data on the public reporting.

24. Purpose of District & SchoolPVAAS Data • Provides users with information to assist them in evaluating the overall effectiveness of a district/LEA or school on the academic progress of groups of students. • This report is NOT a report on teacher effectiveness!

25. What is New? Since Feb 2011 Public Reporting Release: • New: Web-based Scatterplots • New: Color Scheme/Legends • New: Addition of Science and Writing for Grades 9-11 • New: Rollover Text Feature • New: Redesigned School Search Feature

26. Web-based Scatter Plots

27. Accessing Scatter Plots • The scatter plot report can be accessed directly within the orange “Reports” tab

28. Customizing Scatter Plots Users can switch subjects Users can also switch the the X and Y Axis

29. Roll-Over Feature!

30. Website Demonstrationhttps://pvaas.sas.com

31. District and School Value-Added Reports

32. PVAAS Value-added Growth DescriptorsGrades 4-8 Math & ReadingGrades 9-11 Math, Reading, Science & Writing

33. Example of District Value-Added Summary ReportGrades 4-8, Math & Reading

34. Example of School Value-Added Summary ReportGrades 4-8, Math & Reading

35. Where can I find this information on other reports? Public Site: School Value-Added Summary Report Restricted Site: School Value-Added Report

36. Example: Growth Measure over Grades Relative to the Growth Standard • Growth Measure over Grades Relative to the Growth Standard (on the public report) is the SAME as the Growth Measure over Grades Relative to the Growth Standard on the School Value-Added Report (password-protected site)! 0.0 5th Grade Growth 3.2 4th Grade Growth 2 1.6 Growth Measure over Grades Relative to Growth Standard

37. What is the Growth Measure over Grades Relative to the State? • Represents the average growth across the grade levels served between 4 and 8 compared to the average progress of all students in Pennsylvania at the same grade levels. • It is the average academic growth of the district’s or school’s students, compared to the academic growth of students statewide. • Answers the question, “How much did the district/LEA/school impact the academic progress of its students compared to the progress of other students in Pennsylvania in those same grade levels?”

38. Example: Growth Measure over Grades Relative to the State • Growth Measure over Grades Relative to the State (on the public report) is the SAME as the Growth Measure over Grades Relative to the State on the School Value-Added Report (password-protected site)! ? Average Growth over Grades Relative toGrowth Standard for the State 1.6 Growth Measure over Grades Relative to Growth Standard for the school 0.9 Growth Measure over Grades Relative to the State

39. What is Growth Measure over Grades Relative to the Growth Standard for the State? 2.9 Grade 4 State 3-Yr-Avg -1.4 Grade 5 State 3-Yr-Avg 2 .75Average Growth over Grades Relative to Growth Standardfor the State

40. Example: Growth Measure over Grades Relative to the State • Growth Measure over Grades Relative to the State (on the public report) is the SAME as the Growth Measure over Grades Relative to the State on the School Value-Added Report (password-protected site)! .75 Average Growth over Grades Relative toGrowth Standard for the State 1.6 Growth Measure over Grades Relative to Growth Standard for the school 0.9 Growth Measure over Grades Relative to the State

41. Example of District Value-Added Summary ReportGrades 9-11, Math, Reading, Science & Writing

42. Example of School Value-Added Summary ReportGrades 9-11, Math, Reading, Science & Writing

43. Where can I find this information on other reports? Public Site: School Value-Added Summary Report Restricted Site: School Value-Added Report

44. Is it appropriate to compare the amount of progress made by a district/school to another district/school? • Without taking the Standard Error into account, it is NOT possible to directly compare growth measures across districts/schools. • The color-coding of the growth measures does in fact take • into account the Standard Error. • Note the link to the color code legends on each report. • The Average Growth Index found on the School Search • report feature takes the Standard Error into account and allows a more direct comparison across schools.

45. School Search Feature

46. Purpose of School Search • Users can find and view the progress of public schools across Pennsylvania and search for similar schools based on grade levels tested, various demographics, Intermediate Unit (IU) region, and/or county.

47. Which schools are included when I use School Search? • Schools with at least one tested grade in common as the “reference school” you selected. • Example: Your reference school is a grade 6-8 school • Other schools included in the search may include grade 6-7 schools, grade 7-8 schools, K-6 schools, etc. – unless you specific a grade range.

48. Accessing School Search PVAAS School Search is accessed the via links at the top of the page or using the orange Reports tab

49. Predictive Text to Assist in Locating Schools Example: the user enters the word “Pink” and school names containing the word “Pink” appear as options to select.

50. School Search Results