ASER INDIA

1. ASER INDIA Strengthening the process through innovations over time

2. Changes to the process over time Independent process audit • Collective Survey in MP: • Entire ASER team posted in one state for 3 weeks • Many valuable lessons learnt • Introduced “Recheck”: • 2-4 villages to be backchecked post survey in every district • Substantial changes to: • training model • survey model • monitoring & recheck model • Created repository of information to measure data quality • Consolidation & growth • Building on innovations of 2011

3. Lets discuss some of these innovations.. Many substantial and successful measures were introduced in ASER 2011 1. Trainings • Mock trainings • All team members at national and state level practiced training • Helped them revise their content and delivery • b) Rigorous field practice • In all 4 day state trainings, 1 day was spent practicing the survey and 1 day was spent practicing recheck • Helped merge theory with practice for the participants • c)Evaluation & Master trainer (MT) profiling • All MTs and surveyors were given a quiz to assess their understanding of the survey process • Only the ones who passed satisfactorily were allowed to train/do the survey • MT information and performance (in field, in class, and mock trainings) from every state training was documented and results were analyzed. The goal is to create a pool of good MTs who can be relied on each year for ASER and other studies as well

4. ASER 2011 - Innovations 1. Trainings • d) Better training material • Multimedia content used for training • Flex banners – durable; low-cost substitute for projectors in remote districts; allow practice of filling survey formats e) District level training duration now 3 days instead of 2

5. ASER 2011 - Innovations 2. Survey Model • a) “2 weekend” surveys instead of the usual “one weekend” ones • In most districts, 30 villages were surveyed over two weekends in batches of • 15 each weekend • Only 30 volunteers needed instead of 60 – better PTR and hence improved training quality • Increased amounts of monitoring and recheck between the two weekends Cost of ASER increased a fair bit, but so did the quality of survey

6. ASER 2011 - Innovations 3. Monitoring & Recheck • a) Increased magnitude • In ASER 2011, at least 4-8 villages were monitored in every district • Another 4-8 villages were also rechecked in every district • b) “Purposive” Nature • For the first time in ASER, the districts and villages for monitoring and recheck were selected on the basis of certain pre-defined criteria (for e.g. volunteers surveying that village scored low on quiz etc • Feedback from the state teams • In previous years, villages to be rechecked were selected randomly – this purposive re-check was in addition • c) Call Centre at state level • An ASER team member was responsible for regularly telephoning Master Trainers in every district to monitor the progress of the survey • ensured instant troubleshooting of problems and prompt support to remote or problematic districts

7. ASER 2011 - Innovations 3. Monitoring & Recheck • d) Employing mobile technology – the SMS Recheck • Successfully employed in Rajasthan, Gujarat, Chhattisgarh and Himachal Pradesh • MTs texted village related data to a designated phone number • All SMSes were downloaded from a website and data analyzed • Villages for recheck were assigned • Also, provided a sense of the quality of survey at an early stage • e) New recheck method • An extremely detailed recheck method was devised which stopped just short of re-testing the child • New comprehensive recheck format – piloted by entire ASER National team • 25% of time in all state trainings devoted to understanding and practice of new methods • f) ASER Centre Recheck • Entire ASER national team made cross-state visits to conduct rechecks • 14 states and 43 districts were covered in this second round of recheck

8. ASER 2011 - Innovations 4.Repository of Information • e) “District Compilation Sheet” (DCS) • recorded contact numbers, attendance information and quiz performance of all 25,000 surveyors • Data was digitized and information analyzed; this enabled: • i) purposive recheck • ii) Research on data quality – revision to the operational models for ASER 2012 • iii) The data will also be used for dissemination purposes In ASER 2011, approximately 41% of all villages surveyed were either monitored or rechecked by Master Trainers

9. ASER 2012 – consolidation and continuing to innovate • All the successful measures from ASER 2011 have been retained and are being strengthened • Based on the research on data and operations, processes have been tweaked if required Some new features have also been introduced in ASER 2012: • a) Increased duration of state trainings • State trainings are now 5 days instead of the 4 in previous years • More class work, more practice in the field, more effective mock trainings

10. ASER 2012 – consolidation and continuing to innovate • b) Novel use of the District Compilation (DCS) sheet • In addition to all the applications last year, the DCS data will serve as a proxy for the ASER 2012 data • Inferences shall be drawn 2 months earlier than usual; time to investigate learning level trends • Perfect matches in case of other states too!; holds true for Reading levels also! • We’re getting these numbers through SMSes in many states to ensure early receipt of data

11. We’re committed to improving quality through continuous innovation THANKS! All feedback is most welcome