Understanding ‘Learning’ in Pakistan: Building a Comprehensive Picture of School Functioning in Pakistan

1. Understanding ‘Learning’ in Pakistan: Building a Comprehensive Picture of School Functioning in Pakistan MonazzaAslam, Centre for the Study of African Economies (CSAE), Dept. Of Economics, University of Oxford, UK and Idara-Taleem-o-Aagahi, Lahore.

2. Education is Key... • “Education is central to development and a key to attaining the Millennium Development Goals. It is one of the most powerful instruments for reducing poverty and inequality….” (The World Bank); • Access to education is high on the development agenda; • Desire to achieve MDGs has galvanised policymakers worldwide into action. • In many countries progress in school enrolment is phenomenal.

3. ‘Learning’ is critical! • It is now a well known fact – it is not years of schooling that matter to important outcomes but what is actually learnt that matters. • Hanushek and WoBmann (2007) show that differences in learning achievements matter more in explaining cross-country differences in productivity growth than differences in the average number of years of schooling or in enrollment rates. • An effective strategy should focus not solely on sending more children to school but also on enhancing the quality of schooling.

4. Hanushek and WoBmann (2007) show using PISA surveys that disparities in secondary education between developing countries and OECD countries are even larger when one considers not only access but also learning achievements. • Things are not much better at the primary level. In recent surveys in Ghana and Zambia, fewer than 60 percent of young women who complete six years of primary school could read a sentence in their own language. 

5. ‘Learning’ is critical for: • growth and for pro-poor growth; • For poverty reduction; • For reduced inequality; • Fertility, population control; • For access to more lucrative occupations; • Higher earnings during the lifetime; • Women’s empowerment; • Children’s health outcomes and hence future life outcomes.

6. Schooling and ‘Learning’ in Pakistan • While Pakistan has made substantial headway recently in achieving some dimensions of the ‘Education for All’ Millennium Development Goal (MDG2), it still has a long way to go. • although the net primary enrolment rate increased from 45% in 2002 to 54% in 2006 (World Bank), it is pitiably short of universality. • More importantly, the MDG doesn’t focus enough on ‘learning’ outcomes.

7. But... • Enrolment is the very first rung of the ‘access’ ladder. • Meaningful access requires – in addition to school enrolment –good attendance rates, timely progression through grades, and mastery of basic cognitive skills. • Not all enrolled children attend school regularly • not all who attend regularly complete primary schooling; • not all who complete primary schooling acquire adequate learning to be literate and numerate. • Stable and regular attendance of both students and teachers is necessary for learning.

8. What ‘needs’ doing according to common knowledge • Improve curriculum; • Improve teaching; • Improve facilities for ‘learning’; But research on most aspects is limited.

9. What do we know so far?(will talk about work I am associated with) PREVIOUS RESEARCH IN PAKISTAN SHOWS: 1. We know that students in private schools may achieve more and that girls may be worse off in terms of access to private schools; 2. We also know potentially how student A may be targeted through improving weak teaching; 3. ASER (2008) – builds a picture of basic learning and tells us that 65% 5-16 year olds in 11 districts in Pakistan cannot read standard 2 story level text and more than 70 % cannot do two digit division (again standard 2 curriculum). 4. We also know that there are striking gender inequalities in occupational choice and in earnings; 5. Finally, we also know that the labour market rewards women’s ‘skills’ substantially. If today's children are to become tomorrows earning adults, we need to know how they are learning today to ensure they are able to achieve their full potential tomorrow.

10. The private schooling and learning story • Aslam (2009), ‘The Relative Effectiveness of Government and Private Schools in Pakistan: Are Girls Worse Off?”, Education Economics, Vol 17 (3), pp. 329-353. • If you look at raw PIHS data, girls are found more likely to be enrolled in private schools. • But when we condition on household observables/unobs. Picture changes! BOYS are more likely to be enrolled in private schools. • A major channel of lower educational expenditure on girls schooling than boys is because boys are more likely to be sent to private school; • Although average cost of private schools low, relatively more expensive than government school. • Are girls condemned to poor ‘quality’ schooling?

11. Achievement Data used to answer the question: Are girls condemned to poor quality schooling in Pakistan? • Data – one district (Lahore); • School-based sample survey of 65 schools (40 pvt, 25 govt); • Data collected in 2002-2003; • Detailed info on 1800+ students, 300+ teachers – 8th grade students; • Tests administered – literacy, mathematics, ravens (discuss);

12. Raw and standardised Achievement, Reading and Maths scores by school type • Estimates based on School survey conducted by author in 65 schools (40 private and 25 government) in Lahore district in 2002-2003. • A random student with average characteristics would score 9.63 points in maths if in private school compared to 8.10 points in government school (19 % higher) • Predicted reading score of student in private school 8 % higher; Hence girls lose out in terms of the quantity and quality of education acquired! Achievement data helps us identify a major inequality.

13. What is it about teachers that matters for student achievement? • Aslam and Kingdon (2008, RECOUP Working Paper 19, under review). ‘What can teachers do to raise pupil achievement?’ • This study probes the ‘black box’ of teaching to explore the teacher characteristics that impact student learning; • How important are the teaching practices adopted by teachers in determining student achievement?

14. What the study does… • This study takes the direct approach linking teacher characteristics to student outcomes in an achievement production function, but with two innovations. • Firstly, we use a methodology that overcomes the biases associated with the non-random matching of students to schools and, within schools, to teachers. • Secondly, we also test the importance of classroom practices and teaching techniques rather than confining attention only to teachers’ resumè characteristics i.e. qualifications, experience and training.

15. Teacher characteristics data • Standard teacher chars such as age, gender, certification, training, tenure, experience etc. • ‘Teaching methods and techniques’ information gathered by asking each individual teacher to answer a series of questions such as record minutes/week spent on average in giving surprise tests/quizzes, maintaining discipline and order, writing material on the black board etc. • Teachers also given series of questions on use of lesson plans, whether they read out loud from books, how well they engaged students when teaching etc.

16. Methodology • Achievement production function – ‘direct approach’ • Estimate: Aik = α + βXik + δSk + μi + ηk (1) • Where i = individual and k = school • X = individual and family characteristics • S = school and teacher characteristics • μi = individual-level unobservables • ηk = school-level unobservables

17. Summary of results • Resume’ – i.e. teacher qualification, training, etc. - characteristics of teachers don’t matter to student learning – highly controversial claim but supported by studies from India, USA etc.; • Girls benefit from being taught by female teachers; Why? 1) Preferences hypothesis...same-sex teachers prefer same-sex students , 2) stereotypes influence teachers evaluations; We gave tests and marked them so can rule out the ‘evaluation’ stemming from these 2 but can’t rule out possibility that knowledge transmission not affected by either route. • Teaching ‘process’ – lesson-planning, quizzing and involving them in classroom discussions clearly matters to achievement.

18. G and P school differences • We know from recent literature that private schools more ‘effective’ than govt. schools (World Bank researchers, Alderman’s work etc. on primary and my work on Middle school students). • Why more effective? • Private school student body superior quality? • Infrastructure superior? • Teacher quality superior? • In terms of apparent observed charachtersitics, G school teachers more certified, experienced, better paid. • Know nothing about ‘unobserved’ motivation etc. – indeed contract stuctures in G schools generate perverse incentives. • We know nothing about ‘process’ differences or even how the production functions differ across school types.

19. Conclusions of ‘Teachers’ study • Delved into the ‘black box’ representing teaching; • Identification strategy was unique; • Importance of class room practices; • Separate estimates for govt. and private schools; • Teacher unobservables still remain as source of endogeneity undermining confidence in attributing causality to observed teacher variables.

20. Conclusions continued… • Standard teacher resume’ characteristics have no bearing on student’s standardised mark; • Teacher pay schedules are ‘inefficient’; • Girls in 8th grade benefit being taught by female teachers; • Teaching classroom practices (lesson planning, involving students by asking questions, quizzing etc.) substantially benefit student learning. • Private schools teachers more ‘efficient’ and use the more innovative teaching methods in imparting learning.

21. What does ASER (2008) tell us? • Annual Status of Education Report (ASER). • The purpose of the ASER rapid assessment survey and its planned annual rounds in rural areas is twofold: • To get reliable estimates of the status of children’s schooling and basic learning (reading and arithmetic) up to grade II at the district level • To measure change in these basic learning and school statistics over time- annually • In ASER Pakistan 2008 , 16737 children and 6520 households were surveyed in the rural areas in 11 districts

22. How ASER is conducted? ASER is a citizens-led initiative conducted in a campaign mode. This means that ASER is not conducted by any single organization. Instead, ASER is done in collaboration with a variety of civil society organizations, universities, research institutions government organizations and citizens’ groups. Collaborators voluntary supporters, teachers and students devote their time and resources to conduct ASER in their region/districts. Pratham and ASER India, provide technical support at various stages of the survey to the ASER Pakistan.

23. Scale & Scope of Survey Coverage : In all five provinces i.e. in Sindh, Balochistan, Punjab, PakhtonKhawa, GilgitBaltistan, FATA and AJK. Only rural areas will be sampled. Phase I : Year I 2010 – 30 districts across Pakistan Phase II: Year II 2011 – 70 districts across Pakistan Phase III : Years III, IV , V all districts across Pakistan (138 districts) Sample: 600 households per district. Two-stage sample, stratified in the first stage. Obtained by selecting 30 villages per district & 20 households per village.   Villages will be selected randomly using the village directory of the latest Census. The Probability Proportional to Size Sampling (PPS) technique will be adopted as an appropriate one when the sampling units are of different sizes. Focus on nuances between public and private . In each village profile of 1 govt. and 1 private school will be collected .

24. ASER Pakistan Assessment Tools

25. ASER 2008 in Pakistan ASER Pakistan 2008 only completed in 11 districts of 2 provinces: Punjab, Sindh and the Islamabad Capital Territory. Unfortunately, ASER Pakistan 2008 could not be undertaken in Baluchistan and NWFP due to the Oct. 2008 earthquake/floods & rapidly deteriorating law and order situation /conflict ..local collaborators got engaged in multiple emergency responses

26. Districts’ Summary ASER 2008 In ASER Pakistan 2008 , 16737 children and 6520 households were surveyed in the rural areas during the survey in 11 districts

27. Enrolment is still a problem – children aged 5-16 More girls are out of school – no surprise here; Enrolment is the very first rung of the access ladder – we stumble on the very first one.

28. Enrolment by School Type Apparently a larger proportion of girls enrolled in private schools (46% versus 41% boys). This finding is NOT different from what the PIHS data tells you when you do not condition on household observables. If we were to condition on observables etc., I suspect ASER 2008 finding would be identical to Aslam (2009) – girls lose out in the quality of schooling available to them!

29. What about ‘meaningful access’? • Good Attendance rates? • Timely progression through grade? • Mastery of basic cognitive skills?

30. Attendance • Teachers Attendance is over all 78% • Primary level 75%; • Elementary : 82% • Children’s Attendance overall : 74% • Primary level 79%; • Elementary level 72%

31. Think about it! • 79% children attendance in grades 1-5 means that in an average class of 49 children, 10 children were not attending school on the day visited. • If there are 15 teachers in a primary school, 4 out of the 15 teachers are not available on the day visited. • Aslam (2003), ‘Determinants of Student Achievement in Government and Private Schools in Pakistan’, The Pakistan Development Review, 42:4, Part II (Winter), pages 841-876: finds that student absenteeism has a detrimental affect on student learning.

32. School Facilities: Provision & Use • Primary Schools • 31% schools at primary level are without water facility, • 42% schools are without toilet facility • ElementarySchools • 23% schools at Elementary level are without water facility, • 27% schools are without toilet facility

33. What about mastery of basic cognitive skills? • Standard 2/3 instruments designed and all children between 5-16 years tested. • Children found to be performing way below curricular standard.

34. Standard 2 Reading ability of 3-16 year olds Of the 16, 737 children tested, only about 30% are able to read a story with fluency in Urdu or local language.

35. Standard 3 Mathematics ability of 3-16 year olds Of the more than 16,000 children tested, only about a quarter can do double digit division.

36. Learning Levels (Reading) • Learning Ability of the Age Group 5-9 Years: • 27% of the children read para or level - I text • 12% of the children in this age group read story text level-II • Learning Ability of the Age Group 10-12 Years: • 70% of the children in this age group can read para-or level 1 text • 49% of the children in this age group can read story or level II • Learning Ability of the Age Group 15-16 Years: • 84% of the children read para or level - I text • 73% of the children in this age group read story text level-II

37. Learning Levels (Arithmetic) • Learning Ability of the Age Group 05-09 Years: • 20% do subtraction or Level-I(arithmetic) • 6% of the children in this age group solve division (level-II) • Learning Ability of the Age Group 10-12 Years: • 58% do subtraction or Level-I(arithmetic) • 26% of the children in this age group solve division (level-II) • Learning Ability of the Age Group 15-16 Years: • 79% do subtraction or Level-I(arithmetic) • 64% of the children in this age group solve division (level-II)

38. Learning - National

39. Going back to the ‘WHY’ in Pakistan • Research further shows that even the most basic literacy and numeracy skills among adults in Pakistan: • Improve women’s access to more lucrative occupations; • Increase earnings for both men and women once in the labour market; • Reduce fertility; • Improve mortality outcomes; • Increase agricultural productivity. • Etc.

40. Measure ‘basic’ skills • Focus on ‘Public/Private’ /other school types differences; • Rural/Urban divides; • Gender differences; • Socio-economic groups; • School/village location; • Individual level data – allows us to calculate achievement differences b/w different groups;

41. We also need to go BEYOND existing efforts We come back full circle: • Many children still not enrolled in school; • Not all enrolled children attend school regularly; • Not all who attend regularly complete primary schooling • finally not all who complete primary schooling acquire adequate learning to be literate and numerate. • Stable and regularattendance of both students and teachers is necessary for learning (among other factors)

42. Pakistan needs more research • In a new project I am proposing to carry out (with ITA collaboration) two (linked) unique surveys to enable a better understanding of the constraints on (a) children’s schooling participation and (b) student learning, in Pakistan. • Apart from testing children at two points in time in order to measure gain in achievement, the survey will measure factors that might be behind poor child learning but which have hardly received research or policy attention thus far, e.g.: • (i) teacher effort (attendance rate, time-on-task);

43. (ii) teacher competence to teach material in the primary texts (teacher’s knowledge, ability to explain and ability to spot the mistakes that children make); and • (iii) aspects of school functioning such as existence of/ adherence to a school/class time-table; stability/flux in multi-grade class configurations in the school over the year; schooling resources and teacher characteristics; and the extent of community involvement in the school, inter alia.  • The proposed school survey will be unique in Pakistan because it will visit each sample school four times in the school year. This will permit observing the dynamic functioning of schools, in contrast to a snapshot picture.

44. Then what? • Planned interventions can be undertaken once we know the areas that need targeting; • Informed national policy decisions; • Align classroom curricula with actual instruction in class; • Motivate the masses – increase public awareness, mobilize communities, parents, educators, policymakers into ACTION!

45. Importance of having this disaggregated data • Identify the schools/students/areas/groups in most need of educational policy targeting; • Guided decisions on how to allocate resources/mobilize resources; • How to tailor curricula and teaching to specific needs of different groups;