National Assessment Systems:

1. National Assessment Systems: Lessons Learned from International Experience by Luis Benveniste

2. Overview • International perspectives on student assessment • Why have Southern Cone countries espoused national assessment systems? • Conclusions

3. Objectives of Assessments 1. To Promote Educational Quality • by monitoring student learning at the classroom, school, province and national levels • by linking results back to educators and policymakers to inform instructional and curricular reform • by promoting high national standards for student performance • by promoting clear expectations for school level performance

4. Objectives of Assessments 2. To Inform Policy Making • by evaluating the impact of specific policies or interventions • by using results to design compensatory policies • by identifying variables associated with student learning

5. Objectives of Assessments 3. To Foster Greater Efficiency and Accountability • by linking student achievement results to incentives for teachers, schools, districts or provinces • by fostering public accountability and transparency • by informing parents and families about school performance and promoting choice

6. Objectives of Assessments 4. To Inform Student Selectivity • by using tests to place students in a certain track or school • by using results to select who progresses to the next level • by using tests to determine if a student can graduate Singapore Examinations and Assessment Board

7. International Assessments • Comparative perspectives on achievement levels by subject • Analyze effects of country policy differences • Build regional networks to share knowledge and strengthen local capacity

8. International Assessments: SACMEQ • Consortium of fifteen Ministries of Education in Southern and Eastern Africa • Two major cross-national studies and a third under development • Training modules

9. International Assessments: PASEC • Member countries of CONFEMEN—Francophone Africa • Cohort studies of student achievement • Thematic studies—e.g. contract teachers or teacher pre-service training

10. Impact of International Assessments: TIMSS • United States: Lower performance relative to Asian countries in 1995, 1999 and 2003 math and science • “A curriculum that is a mile long and an inch deep”—Lack of opportunities for children to “construct meaning” • US vs. East Asian educational model: collaborative teaching, in-depth conceptual curriculum, parental involvement • Results provided momentum to curriculum overhaul and standards-based reform

11. Overview • International perspectives on student assessment • Why have Southern Cone countries espoused national assessment systems? • Conclusions

12. Southern Cone Countries

13. National Assessments: Comparative Overview

14. National Assessments: Comparative Overview

15. Political Context: Chile • Political context at inception of testing • Municipal decentralization and privatization (voucher system) • Promote consumer accountability • Promote market competition and choice

16. Political Context: Chile • Assessment characteristics: SIMCE • Universal test in Grades 4 and 8 • Public dissemination of school level outcomes • Parental outreach strategies • Rankings by school (within socio-economic banding) • Promotion of school level responsibility over testing outcomes

17. Political Context: Chile • Linking test scores to education resourcing • Targeting the Poor  Assessment results as an avenue for channeling compensatory funding (materials and technical assistance) to low performing schools (P-900 Program) • Providing financial incentives to improve teacher performance  National Evaluation System of School Performance (SNED) awards salary bonus to teachers in high performing schools

18. Targeting the Poor:Chile’s P-900 Program • Equity focus -- Compensatory program for lowest scoring (900) schools in the SIMCE test • Open to public and subsidized schools • Three-year formal commitment • Technical assistance + educational resources • Annual Action Plan

19. Targeting the Poor:Chile’s P-900 Program • SIMCE as a starting point for diagnosis • Comprehensive approach • Teacher professional development • Targeted support to students at-risk • Pedagogic counseling and guidance • Distribution of educational materials and classroom library • Integration of classroom and home environment • Creation of School Management Team

20. Targeting the Poor:Chile’s P-900 Program

21. Incentives for Performance: Chile’s SNED National Evaluation System of School Performance (SNED)  • SIMCE assessment data as a means to: • Link improved teacher performance to teacher compensation • Reward school practices which contribute to student learning • SNED award is equivalent to a one-month salary

22. Incentives for Performance: Chile’s SNED • SIMCE accounts for 65% of SNED score: • “Effectiveness” – Raw SIMCE score in Math and Spanish • “Value Added” – Average SIMCE score gain • Other parameters: • Equality of opportunities (retention rates; integration projects) • Working conditions (full staffing; substitute teachers) • Initiative (pedagogical activities, school development plan) • Parent-teacher integration

23. Incentives for Performance: Chile’s SNED • Financial awards directed to entire teaching team to encourage collaboration • Address discrimination against disadvantaged students • Clear criteria established on expected behaviors and desirable school characteristics • Schools compared to others with similar geographic and socioeconomic characteristics • Fair and transparent system

24. Promoting Consumer Choice and Accountability: Chile • Public dissemination of school level outcomes • Parental outreach strategies through media • Rankings by school (within socio-economic banding) • Promotion of school level responsibility over testing outcomes

25. Political Context: Uruguay • Political context at inception of testing • Centralized educational system • Assessment as a means to pursue “consolidation of social equity” • Reference to Argentine and Chilean experience assessment practices

26. Political Context: Uruguay • Assessment characteristics: UMRE • Universal test in Grade 6, but controlled sample for Grade 3 • Confidentiality of school level data • Teacher outreach strategies • Focus on equity and socio-cultural variables rather than student achievement scores per se • Accent on State responsibility over testing outcomes

27. Uruguay:Building Consensus • Teacher Unions’ concerns: • Categorization/stigmatization of schools • Usage for teacher incentives/punishments or merit pay schemes • Promotion of private over public education • Promotion of “market-oriented” reforms

28. Uruguay:Building Consensus • Educators’ concerns: • Concerns over “external” organization and application • “Competition” to the inspectorate • Appropriateness of standardized/multiple choice instruments • Concerns over “fairness” of using same test for all SES contexts

29. Uruguay:Overcoming resistance “When UMRE appeared, we had a brick on each hand. I was ready to kill them. I had all my justifications against them ready. Little by little they convinced us. Now, after all that has happened and as we get more results, they convince us even more. It is OK that the test is obligatory. It has been a valuable experience.” A representative from the Association of Private Education Establishments

30. Uruguay:Government’s strategy • Focus goals of assessment system • (Re)tailor national assessment priorities prior to launching UMRE • Espouse a participatory approach • Extensive prior information campaign • Involve educators in design process • Creation of a diverse Advisory Board

31. Uruguay:Government’s strategy • Managing the “stakes” of assessment • Emphasize State responsibility in education quality • “Limited” distribution of school scores • Create a climate that focused on pedagogical dimension of testing • Mobilize inspectorate as support network to foster pedagogical reflection

32. Uruguay:Government’s strategy • Contextualize student achievement in relation to socio-cultural variables • Equity objectives: Link assessment to specific compensatory/support activities • Focus on lessons from “effective schools” from low SES contexts • Leveling playing field between public and private sectors

33. Very Favorable Medium High Medium Low Very Unfavorable Total Highly Satisfactory (scores above 80%) 21.0 8.4 3.4 2.0 6.8 Satisfactory (scores 60% to 80%) 45.6 35.3 23.2 15.7 27.8 Unsatisfactory (scores 30% to 60%) 30.6 49.7 60.7 64.4 54.5 Very unsatisfactory (scores below 30%) 2.8 6.7 12.7 17.9 10.9 Total 100.0 100.0 100.0 100.0 100.0 Uruguay:Presentation of Results Percentage of Students by Performance in Math and Sociocultural Context

34. Overview • International perspectives on student assessment • Why have Southern Cone countries espoused national assessment systems? • Conclusions

35. Pitfalls of Testing Risks of High-Stake Scenarios: The stigma of results • Poor results lead to frustration among teachers who may feel disempowered to bring about change • Stigmatizes low performing schools, leading to the "flight" of better students, leading to further segregation in educational services

36. Pitfalls of Testing Risks of Low-Stake Scenarios: Educators pay little attention to the test • Assessment “comes and goes” • No changes in school or classroom behavior • "Lost opportunity" to reflect on student achievement, classroom pedagogy or school administration

37. Some Concluding Remarks • High-stake assessment systems: • Foster public accountability • Performance incentive systems • Prompt close alignment with tests • Motivate changes in school/classroom behavior • But … may also have negative consequences, such as “teaching to the test” • Prompt political “resistance”

38. Some Concluding Remarks • Low-stake assessment systems: • Limited accountability • No direct consequences vis-à-vis test results • Looser coupling between ed practices and tests • Can weaken “backwash” effect of testing • Onus is on Government to foster greater test data use, leading to changes in teaching and learning

39. Good Assessment Systems … • Underscore educational goals desired • Highlight challenges in student achievement, classroom pedagogy and school administration • Provide a framework to map change, design necessary interventions and redirect resources at school and classroom levels

40. Teacher comments: Uruguay • On the basis of the [exam results], we developed a plan for the following year. For instance, the discussion over problem resolution was very important for us in order to go deeper into this issue, to work more on reasoning. I don’t know if this took us further away from the [official curricular] program, but ... Also, we’ve been working on the language curriculum in teacher meetings. In these sessions we analyzed some of the test items.

41. Thank You