Ernesto Treviño Senior Advisor LLECE September 21th, 2011- Cartagena de Indias, Colombia

1. Learning assessment among indigenous students in Latin America: Challenges of measuring, interpretation and study of factors associated to learning y context of cultural diversity and social inequality • Ernesto Treviño • Senior Advisor LLECE • September 21th, 2011- Cartagena de Indias, Colombia

2. Agenda • Background • Use of standardized tests in contexts of cultural diversity • Purposes and uses of evaluation • Tensions between standardized evaluation and cultural diversity • Knowledge, teaching and learning among indigenous groups • Validity, culture and equity when measuring learning among indigenous populations • Equity and measuring in indigenous contexts • Factors associated to learning and intercultural bilingual education • CIPP model for Latin America and the Caribbean • CIPP model and intercultural bilingual education

3. Background • Latin America • Deep social inequalities • Marginalization of indigenous groups • Stereotyping and prejudice of indigenous groups • Constitutional recognition of cultural, ethnic and linguistic diversity in several countries • Expansion on the application and use of standardized tests without necessarily considering this diversity • Challenges – Peru assesses four indigenous languages

4. Use of standardized tests • Tension between standardized evaluation and cultural diversity • Recognition of diversity • Extensive use of standardized evaluations • Admission tests to different educational levels may seriously affect educational opportunities • Limited scientific and educational assessment development for considering cultural diversity • High levels of investment for developing tests • Disagreement on the writing of indigenous languages • Lack of specialized personnel

5. Use of standardized tests • Purposes of standardized tests • To know the levels of student achievement in different areas (low stakes or no consecquences) • To make decisions that affect directly individuals or institutions based on the results of the tests (high stakes or with consequences) • Uses of standardized tests • Monitoring public policy • Program evaluation • Identification of schools that require support • Uses with problems of causality • Asignation of incentives and penalties • Stimulation of competition among schools • Social sanctions for low performing schools

6. Use of standardized tests • Knowledge, teaching and learning within indigenous groups • Transcultural psychology • Formal education: • Individual work • Analytic thought • Intensive language use • Strict time framework • Learning of scholastic contents • Individual evaluation of students

7. Use of standardized tests • Knowledge…(continuation) • Teaching and learning in indigenous contexts: • Collectivist vision of social organization, teaching and learning • Culturally appropriate forms for participation on learning • Pedagogic strategies • Performance measurement • Holistic thought • Observational strategies and practical participation for learning • Lifelong learning and skill acquired through practice • Non interventionist attitudes of adults • Room for mistakes • There is no possibility of public failure: shared responsibility for results

8. Use of standardized tests • Validity, culture and equity when measuring learning among indigenous populations • Validity • Property of the inference • An inference is valid if it is alinged with the aims of the tests • Validity evidence • Test content • Answering process • Internal structure of the test • Relationship with other variables • Consequences of the test • Cultural validity • Cultural characteristics for interpreting the world, solve problems and impact results • Universality of constructs

9. Use of standardized tests • Validity…(continuation) • Bias • Systematic difference in the validity of the inference, in a way that it is less valid for one (or several groups) under comparison than others • Systematic difference is necessary evidence, but not sufficient for demonstrating bias • It is necessary to also have substantive evidence of test contents • In Latin America indigenous students show systematic differences that affect their results • Cultural and language bias in standardized tests • Adverse impact • Test may be biased but not necessarily disproportionally affect low achievement groups

10. Use of standardized tests • Validity…(continuation) • Tension between diversity and equity • Intercultural and bilingual education (IBE) and society • Dominant language is the vehicle for educational mobility • Social structures value the dominant language • Advancement in language normalization • Do not neglect discrimination

11. Use of standardized tests • Equity and measurement in indigenous contexts • Latin America the most unequal region in the world • Use of standardized test with consequences • Measurement for policy feedback and program evaluation • Evaluation for improvement • Both standardized and classroom evaluation

12. School Context Education context at home Socioeconomic and cultural index Gender Indígenous populations Child labour Input School infrastructure Access to basic services Availability of computers Additional teacher work Learners Attendance at pre-primary education Repetition Product Learning Civic education Social and economic participation Process School climate School management Teacher satisfaction Teacher prestations Factors associated to learning in Latin America CIPP Model UNESCO, 2010

13. Demographics (Indigenous vs. Non-indigenous) • Population • Household size • Gender distribution • Age distribution • Socioeconomic information (Indigenous vs. Non-indigenous) • Income • Quintil of autonomous income • Educational indicators • Illiteracy rate • Years of schooling • Educational coverage • Number and size of schools • Number of teachers per school • Student-teacher ratio • Sociolinguistic Context • Ethnic auto-adscription • Number of • Competent Indigenous-speaking people • Learning in indigenous language • Indigenous language use • IBE acceptance • Principal perception of parental disposition • Teacher perception of parental disposition • Parental attitude/cognitive disposition • Parental attitude/affective disposition • Parental attitude/conative disposition Factors associated to learning in IBE: Context

14. Pedagogic resources • Texts: • School/non-school • Form of ellaboration • Cultural contents • Didactic material, audiovisual resources in language • Computers • Initial teacher training • Teacher level of education • Training and skills in cultural contents • Tranining and skills in indigenous language • Training and skills in didactic of the cultural contents • Training and skills in didactic of indigenous language • Training and skills in didactic of dominant language • Monetary resources and school institution • Monetary resources: quantity of funds available, timely distribution, flux of delivery process • Curriculum adaptation: school project, local curriculum • Actors involved in curriculum creation and adaptation • Role and support from IBE educational authorities Factors associated to learning in IBE: Input

15. Teaching practices • Classroom practices • School organization: teachers work on IBE and update of local curriculum • Implementation of teaching second languages and intercultural perspective • Community-school mediation • School management • Internal: instruments for curriculum adaptation, committment and motivation • External: relationship with educational authorities • Relationship with community (level of inclusion in curriculum adaptation and implementation) • Community-school relationship • Participation in local practices • Celebration of local traditions in school • Community participation in curriculum adaptation and implementation Factors associated to learning in IBE: Processes

16. Learning • Oral communication in indigenous language • Written communication in indigenous language • Effect in other learning outcomes • Intercultural practices in the school • School community values ethnic diversity • Visibilization of interculturallity in school • Community participation in IBE programs • Harmonic relatiohshps between indigenous and non-indigenous • Valued indigenous culture • Improvement in indigenous students self-esteem • Increase in the areas of use and promotion of language( public spaces, school, etc.) • Languages revitalization • Number of indigenous languages speakers (by age, gender and geographic location • Intergenerational • Recovery or loss of mother language Factors associated to learning in IBE: Products

17. Conclusions and recommendations • Standardized tests • Inferences related with the purpose of the test • Useful to know the level of achievement of the school system • Help in monitoring programs and policies • Problems in culturally diverse contexts: • Validity • Bias • Adverse impact

18. Conclusions and recommendations • Educational evaluation • Cultural and linguistic differences between indigenous and non-indigenous groups • It has neither incorporated culture nor indigenous language • More extended use of high stakes standardized tests • Should we test in indigenous languages regardless of the fact that it may not be the language of instruction? • Factors associated to learning • Shared factors in indigenous and non-indigenous settings • IBE • Different objectives and measurement of achievement • Insufficient empirical evidence on factors

19. Thank you!ernesto.trevino@mail.udp.cl