GEMS: G rowth & E ffectiveness M easures for S chools

1. State of Israel Ministry of Education Culture & Sport Office of the Director General Division of Evaluation & Measurement GEMS: Growth & EffectivenessMeasures for Schools Formative Assessment for Schools

2. Gems is an aid to management for school principal and teachers • Gems’ major purpose is to aid school staff in making data-based decisions, in order to use resources wisely and to improve school functioning • Gems is a school oriented assessment elucidating the strengths and weaknesses of the school and thus enabling it to allocate resources where most needed • Gems perceives the school as an holistic system

3. Teacher-student relations Learning environment Staff development Achievements Organization Curriculum Educational plans Teacher-Parents relations Teacher-Teacher relations

4. Gemsindicators were based on requests by 200 school principals and 200 supervisors when asked what reliable information they need in order to improve school functioning • Gemsincludes the following indicators: • Pedagogical Environment • Action Plan • School priorities and staff consensus • Teaching resources & teaching methods • Weak students and the help they get • Academic Achievementsin 4 basic subjects (Language, Mathematics, Science, English) • School Climate and Work Environment

5. GEMS is administered each year in 50% of elementary and junior high schools, so that each school participates in GEMS once every two years • GEMS sources of information: • Tests for 5th and 8th grades in Language, Math, Science and English • Questionnaires for all students in 5th through 9th grades dealing with school climate, pedagogical environment etc. • Interviews with all principals and teachers dealing with work environment, teaching methods, staff development etc. • Gems is administered at the beginning of the school year so that schools may get the report during that same year, and shorten the possibly stressful period of anticipation of the assessment

6. GEMS is school oriented. Its main goal is not to provide a national “state of the union message”, but rather to enable schools themselves to plan rationally and to follow up, over time, the fruits of their efforts • However, the data also serve the staff of the Ministry of Education and its units. Thus, over 20 different aggregate reports are prepared each year to help policy makers in different branches of the Ministry to make data-based decisions

7. The rationale of GEMS is that a necessary condition for improving the educational system is to improve the interaction between teacher and students on the class level • Such an improvement is depends on schools basing their plans and practices on continuous, accurate, reliable, longitudinal data • In order to plan and make data-based decisions schools have to take the following steps: • to set specific goals • to know were they are • to take actions directed to the goals • GEMS is a tool facilitating these steps

8. Setting Specific Goals

9. Goalshave to be specific in order to be measurable and to avoid contradiction between goals • For example if the goal is “enhancing academic achievements” assessment must specify at least 4 indicators: • Average grade • percent of failures • percent of distinctions • difference in grades between students from deprived SES and those from high SES • As shown below these are not necessarily identical:

10. Goal set for schools: raise average by 10%

11. How schools operated to reach the goal?Does assessment distinguish between them?

12. “Without data you are just another person with an opinion”Knowing where we areAfew examples of data for a specific school

13. School priorities: consensus? School Example • Principal: Achievements • Teachers: • language skills • school climate • violence prevention • Math • Arts • school climate • English • Respect for teachers • Keeping school clean • Independent study • Parental participation • “Closing the Gap” • language skills • Don’t know National data

14. Weak students Teachers: reasons for learning difficulties weak students

15. 2004 2002 Students’ Achievements

16. Assessing Change in 2004 In order to evaluate the changes in students' achievements, the following analysis was undertaken: comparing the change in a given school (over the two years) to the change found in a Comparison Group, consisting of schools with same background characteristics and same starting point in 2002. The school may thus draw conclusions such as: • Achievements improved more in the school than in comparable schools • The change in the school is similar to the change in the comparison group • Achievements in the school show less improvement than that in other, similar, schools

17. 2004 2002 Percent of students that didn't meet standards 20% to 25% of students have private tutor paid by parents

18. Achievements by SES -- 2004 The gap in achievements between the two groups is ½ to 2/3 standard deviation

19. Teaching methods and Homework -- 2004 • 10% of the teachers coordinate amount of homework • Students are required to prepare 25 minutes homework for each hour in school • Homework is mainly drill of material taught in class • 10% of the teachers require learning new material or looking for sources • 60% of teaching time is frontal teaching

20. 2004 2002 Student perception of teachers’ attitudes

21. 2004 2002 Teachers’ perception of the work environment

22. Taking actions to reach goals: Action Plan We are free to choose our actions, . . . but we are not free to choose the consequences of these actions.Stephen R. Covey (1996)

23. ? What to diagnose? ? indicators derived from previous data indicators derived from school goals To define specific indicators and to diagnose school relative to goals

24. GEMS ? ? CLASS SCHOOL To analyze GEMS data on two levels

25. To ask questions • Every school supervisor, principal, teacher should have a dialogue with available data (from GEMS and from school internal evaluations) and to ask himself: • Did I know… • Did I ask myself… • Did I check…

26. To plan according to dataAn example: what should be done in classes with different composition Class B Class A

27. Prepare school action plan and define (specifically)where am I and where I want to be…

28. evaluating Working Planning

29. Feedback to assessment from 1,036 principals: 2004