What is KEYS and What Makes it Unique as a School Assessment and Improvement Process?

1. What is KEYS and What Makes it Unique as a School Assessment and Improvement Process? Sonia Jasso Yilmaz NEA Priority Schools Dept. Center for Great Public SchoolsAugust 13-14, 2013 syilmaz@nea.org

2. KEYS Session Objectives To provide you with the KEYS information you will need as KEYS “advocates”-- educators and affiliate leaders and staff who provide assistance and support to KEYS as a strategy for union-led school transformation To understand Why KEYS? And What is KEYS? To understand What Makes KEYS Unique among school assessment and improvement programs

3. KEYS 2.0 What Have You Already Heard,or Know about KEYS?

4. Why KEYS? Increases student achievement for ALL students in ALL types of schools (thru instrument & processes Δ’s) Supports NEA’s efforts to focus on student achievement, help schools meet ESEA/NCLB requirements and address the achievement gaps Makes ALL schools high quality schools Helps schools address the need for data-driven-decision making (3DM) Creates opportunities for association-district cooperation Supports the Association’s advocacy and membership recruitment & retention efforts

5. A Quick Brainstorming Activity “What are the essential elements or conditions that characterize a high performing, high quality school?”

6. A Quick Brainstorming Activity (15 minutes approximately) Quickly volunteer to be (or pick) a scribe who will record the responses from colleagues at your table Chart out the essential elements/conditions of a high quality/performing school—what first comes to mind? Individually: move to each table’s easel to see results-- a quick “Chart Walk” continued next slide

7. A Quick Brainstorming Activity Each Table: Share your impressions of what you saw on “Chart Walk” with colleagues at your table As a large group: Let’s look at the “42 Indicators” sheet— It’s composed of 6 Keys and 42 indicators; Did anyone identify an element not listed? Each Table: One person share an observation about findings with the Large Group

8. The NEA Six “Keys” of School Quality The 42 indicators are clustered into six essential “Keys” that define the standards for school quality: 1. Shared Understanding and Commitment to High Goals Big Picture: Distributive & Effective Leadership(5 Indicators) 2. Open Communication and Collaborative Problem Solving Big Picture: Relationships Among Stakeholders (9 Indicators) 3. Continuous Assessment for Teaching and LearningBig Picture: Knowledge-Based Decisions (5 Indicators)continued

9. The NEA Six “Keys” of School Quality 4. Personal and Professional Learning Big Picture: Knowledge and Competence (11 Indicators) 5. Resources to support Teaching & Learning Big Picture: Use and Availability of Resources, Supports, Services (5 Indicators) 6. Curriculum and Instruction Big Picture: Quality and Depth(7 Indicators)

10. What is the NEA KEYS 2.0 Initiative? Deliberative process for continuous school improvement Research-based and data/information-driven Supports the school’s and district’s efforts to build internal capacity to improve and be accountable Begins with the use of a school assessment tool--an online survey--completed by the school community key stakeholders that yields scores on 42 indicators of school quality that correlate with high student achievement

11. Why KEYS? What is the NEA KEYS 2.0 Initiative? Questions?

12. NEA KEYS 2.0 BREAKNext: Why is KEYS Unique?

13. Our KEYS 2.0 Top 10 List Ten Reasons Why the NEA KEYS 2.0 Initiative is Unique!(Notin Order of Importance)

14. REASON # 1KEYS Theoretical Roots Organizational Quality (organizational conditions):W. Edward Deming Effective Schools Research Curriculum & Instruction: Newman and Wehlage (University of Wisconsin) APA’s Learner’s Centered Principles (Framework for School Reform) Charlotte Danielson’s Framework for Teaching National Board for Professional Teaching Standards (NBPTS) standards Professional development, learning communities, trust relationships: Milbrey Mclaughlin (Stanford), Judith Warren Little (UC Berkeley), Ann Lieberman (Columbia), Michael Fullan (University of Toronto), Anthony Bryk (Stanford) and the NSDC (now Learning Forward) professional development standards

15. REASON # 2The Empirical Validation Research Based: A Three Phase Process:--The Advisory Groups --The Development of the Instrument --The Analyses of the Data

16. REASON # 2The Empirical Validation NEA KEYS Advisory Group: Michael Fullan, Willis Hawley (U of MD), Ann Lieberman, Joe Murphy (Vanderbilt), Jomills Braddock (U of Miami), Susan Moore Johnson (Harvard) KEYS 2.0 Research Advisory Group:Jomills Braddock, Mark Smylie (U of Illinois, Chicago), Floraline Stevens (NCES), Helen Marks (Ohio State), Sylvia Rosenfield (U of MD)

17. REASON # 3Highly Credible with Members Ownership of the process and the data KEYS surveys are completed anonymously and results are strictly confidential Schools decide what to do with the data The survey is available in English or Spanish

18. REASON # 4Inclusiveness The whole school community can participate, including family/guardian/parents and community members, using the KEYS family and community surveys New Student survey (Middle and High School) is now available Separate analyses are provided for each constituent group allowing for comparisons of perceptions among the groups

19. Reason # 5Objective Baseline Data and Targets School average and standard deviation scores are provided for each of the 42 quality school indicators Normative and longitudinal comparisons can be made

20. Reason # 5Objective Baseline Data and Targets School average and standard deviation scores are provided for each of the 42 quality school indicators Normative and longitudinal comparisons can be made

21. Reason # 6Sets Stage for Further Analysis and Reflection Opportunities to discuss difficult issues that would otherwise be muted (the data speaks) Opportunities to collect and use data forestablishing priorities and making more rational school improvement decisions Opportunities for promoting learning communities and encouraging essential conversations

22. Reason # 7School & District Results are Available School and District reports are can be accessed online and printed School data and normative comparison data are reported separately for each of the 42 quality indicators In the School online data report, results are linked to KEYS resources

23. Reason # 8NEA and State Affiliate Supports NEA supports KEYS 2.0 through a variety of resources and continued technical support in the context of action research and continuous school improvement State Affiliates in several states provide support to their local associations implementing KEYS

24. NEA Supports—Other Elementsof KEYS 2.0 Initiative KEYS trainings—Introductory and Advanced NEA KEYS Tool Kit – Migrated to the Website Action Guide and Facilitation Guides Quality schools infrastructure: NEA KEYS Coaches KEYS Spanish language surveys KEYS Networking & Web-site/Online Meeting Spaces NEA KEYS online school improvement resources (Data reports, CSI model and Facilitator’s Toolkit)

25. KEYS 2.0 NEA Trainings KEYS Orientation/Readiness KEYS Basic 101 KEYS Basic data analysis and action planning KEYS Advanced data analysis and use of quality tools KEYS Coaches Training KEYS 24/7 Technical Assistance (web concerns) KEYS Web Tutorials and Resources

26. Reason # 9Cost Sharing Factors Other than in-kind school and district supportsfor NEA affiliated schools and districts, at this time, there are no costs associated with use of the NEA KEYS Initiative As a school assessment and improvement initiative, KEYS’ features, user supports and online resources make it unique among programs in the market that cost thousands of dollars

27. Reason # 10KEYS+ KEYS is more than a school and district assessment system KEYS as a set of standards for measuring school conditions for teaching and learning KEYS as a set of tools for helping schools assess their strengths and weaknesses and build capacity for school improvement KEYS as a measure of internal school accountability

28. Summary: Why is KEYS Unique? KEYS is a comprehensive, research-based, and data driven deliberative process for continuous school improvement It is easy to use and is web-based: www.KEYSonline.org Begins with a series of school assessment surveys (English or Spanish) completed by the different stakeholders in the school community 4. Provides objective baseline data on each of the 42 indicators of school quality and allows for comparisons among stakeholders

29. Summary: Why is KEYS Unique? Highly credible with staff and other stakeholders-it provides a “safe” voice re: teaching & learning conditions and promotes a sense of ownership and empowerment 6. Data reports, supports, and resources are provided online as well as face-to-face trainings 7. Supports internal/system capacity-building

30. Why is KEYS Unique? Questions?

31. KEYS Data Analysis: An Introduction Sonia Jasso Yilmaz NEA Priority Schools Dept. Center for Great Public SchoolsAugust 13-14, 2013 syilmaz@nea.org

32. A Challenge How do you know whether (or not) the key elements you identified earlier existin your school? How do you know whether these elements exist at a level that is sufficiently highto make a difference? How would these elements be measuredor quantified? And, if they cannotbe measured, how would you know if improvement were made?

33. What KEYS Measures The KEYS survey yields scores on 42 indicators of school quality that correlate positively with student achievement KEYS Results are reported in six Keys that offer a vision and define the standards for school quality

34. Accessing KEYS Surveys@ Website From Face Page: www.keysonline.org LiveSurvey Enter Enter Live Survey Demo Survey

35. Accessing KEYS Demo on the WebsiteAfter clicking on “Take the Survey” on the face page, you see this screen for the entireKEYS Demo, including practice surveys Click on “Preview”

36. KEYS Website Link Access to Surveys:http://www.keysonline.org/about.htmlAccess to the Reports Page: http://www.keysonline.org/results/index.html?school_id=826

37. KEYS Results Reports • Staff Report: displays aggregate of school respondents' scores for each of 42 KEYS Indicators and it points to various resources • Family/Parent Report: displays aggregate responses for each question, clustered under 8 themes with comparison to staff • Community Report: assesses and displays a subset of the questions from the parent survey, clustered under 6 themes • Student Report (new): displays aggregate student responses to a set of questions clustered under 20 different categories • Special “Theme” Reports: display KEYS results composed of indicators from more than one Key around alternative themes: • Collaborative Culture • School as a Learning Organization • Standards-Based Curriculum, Instruction and Assessment • Data-Based Decision Making • Closing the Achievement Gap • District Report: displays aggregate of school responses and distribution of school responses for each of 42 indicators

38. Understanding the Graph for each Key Labels Vertical Axis Indicators (groups of questions that measure the same concept) KEY 1. Shared Understanding and Commitment To High GoalsRespondents Provide Direct Instruction to Students Horizontal Axis Measure of perceptual quality: 5 point scale Left side (low value for the indicators) Right side (high value for the indicator)

39. Understanding the Graphs for each Key Labels SchoolAverage All Schools Average Standard Deviation 90th Percentile Score • Data Points • School average (Black) • All schools average (Red) • 90th percentile score (Yellow) • Length of the horizontal bar • (Blue/Purple) 1 standard deviation above and one below the • school average (measure of • agreement or consensus) • The Goals for your school in terms of continuous improvement for each indicator • The school average moving • continuously toward the right side • (agree – high value of quality for • that indicator) • At the same time, reduce the • standard deviation (narrow the • length of the horizontal bar, • meaning greater agreement among • respondents)

40. Hierarchical organization – Keys—Indicators—Items/Questions Rating- the degree to which the respondents believe that the indicator accurately describes the school: average of all respondents on the questions that make up each indicator 1 = strong belief that indicator does not describe the school (Very Low or Poor) 2 = some belief that indicator does not describe the school (Low or Poor) 3 = Neutral 4 = some belief that the indicator describes the school (High or Good) 5 = strong belief that the indicator describes the school (Very High or Good) KEYS Vocabulary

41. KEYS Vocabulary(Continued) Consensus- level of agreement among respondents on the rating for each indicator • Low consensus = wide purple band • High consensus = narrow purple band

42. KEYS School Level Staff Reports: An Overview • All Indicators Shown For Each Key • All Indicators Shown For Each Key/One Key at a time • One Indicator at a time, with resources • Survey Questions that comprise an individual Indicator • “Next Step” Resources • All Indicators for One Key--Multiple KEYS Runs

43. Staff Report: All Indicators KEYS School LevelStaff Survey Results Reports

44. Staff Report: One Key at a Time Staff Survey Results Reports

45. 1.1 Staff Report: One Indicator at a Time (1.1), With Two Types of Resources Staff Survey Results Reports(sample)

46. Sample: Survey Questions for KEYS Indicator 1.1 KEYS Survey Questions: How Used in KEYS Data Reports

47. KEYS Results Reports

48. Staff Report: All Indicators for One Key-- Multiple KEYS Runs KEYS Results Reports

49. KEYS Data Analysis Questions?