2C/D Approach to Sampling/ Surveying Administering the assessment/survey

1. 2C/D Approach to Sampling/ SurveyingAdministering the assessment/survey

2. UNICEF/ UIS Initiative • 25 countries (including Brazil, Cambodia and Zambia) engaged in initiative • Nearly all country reports use administrative data and standard household surveys to report on numbers of children falling within each dimension • Complementary surveys in Congo: • Children in facilities from defined lists • Street children usually interviewed at night

3. ASER and UWEZO • Both employ citizen volunteers • ASER survey is in rural areas; and they recognise that • UWEZO has a cascade training model based on 14 master trainers with 7,620 volunteers

4. Recommended Approach (1) • List, including ages of household members; then • Interview all households reporting a 15 year old, after making an appointment for a revisit • In most countries, about 3 times as many household will be revisited as ‘necessary’; but • Alternative to ask precise age and status vis-a-vis school of teenager could be unreliable • Revisit with a three part instrument: • Filter to verify reported age and establish her/his precise status vis-à-vis school • Test • Contextual questionnaire

5. Recommended Approach (2) • Crucial cross-checking reported age • Be careful about tendency for responses to behaviour or situations that respondent believes to be disapproved to be biased • For non-household populations, distinct samples should be drawn from each group • None of other surveys has faced problem of focussing on a specific age group

6. Ensuring high Response Rates • Many OOS 15 year olds may be uninterested in being tested or interviewed • Caregivers could help members • Quite large proportions are working for or with other family members • Also, based on recent DHS surveys, most appear to be living in parents’ households (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia)

7. Questions for discussion • How to administer the assessment/survey? • What can we learn from other experiences (ASER/UWEZO/Other)? • What do you think of the recommended approach? • Any other alternatives to be considered?

8. Options for administering the test

9. Persuading OOS 15 year olds • OOS 15 yos may have a negative attitude towards school—type activities (such as testing) either because they have recently left or because they are antagonistic to their peers still in school • Possible ‘encouragements’ may come from other members of the household. • Quite large proportions are living in their parents’ household, (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia); although this may be countered by equally large proportions who are working for / with a household member (who may not want to release the 15 yo for interview and test)

10. Possible Role of Incentives • Incentives often used in developing countries • Recognised that, in developing countries, not always seen as appropriate (LAMP) • Nevertheless, where only OOS 15 yosare targeted, refusals will be problematic, so important to maximise response rate • None of surveys in developing countries have used incentives and all appear to have obtained response rates of >95%, relying on peer pressure from citizen volunteers • BUT, they were interviewing all households

11. Incentives are Problematic • We do not know how they might affect responses, they may have unintended consequences, and could be quite expensive • If incentives are considered, Singer and Ye (2012) recommend: • Offer small prepaid incentives to all sample members; this will increase sample size and help satisfy fairness criterion • Offer differential incentives to those who refuse (or a subsample) for bias-reduction reasons

12. Incentives: Recommendations In the context of this exercise, there are two other issues: • Should the incentive be offered only to the 15 year old or to the caregiver/ employer or to both? • If citizen volunteers are used, it will be difficult not to give them an incentive as well. • In any case, incentives should be explored in the field trial

13. Interviewing and Administration • Tests carried out by ASER, OOSCI and UWEZO all by citizen volunteers, who had been trained by professional trainers. • Tests carried out in children’s households, with exception of complementary surveys in institutions and of street children • LAMP assessment also household-based • Demonstration that most 15 yos were working for a family member validates this approach • Mobile phones have been used to collect high frequency panel data (Dillon,2012); and household survey data (Tomlinson et al 2009)

14. Suggested Approaches • Place and Time of Testing: in most cases at their home and probably in the evening, although that may pose problems of security • Recruitment, Selection, Training, Deployment of Interviewers: from locality, sensitive to teenage attitudes! and very flexible. They may have to work in pairs, especially after dark • Mode of Testing: possibility of designing test so that it can be administered by cell phone should be explored by contractor

15. Cost and Logistical Implications • Assuming a two-stage approach is adopted, and final sample of 100 OOS 15 yos is required in each of 10 geographically dispersed locations in each country, then: • Recruitment, training and monitoring of citizen volunteers to identify c.3,000 households reporting a 15 yo • Professional interviewing of 300 households to administer test to c.100 OOS 15 year olds • Additional context-specific costs for sampling non-household groups

16. Questions for discussion • How difficult will it be to ensure a high response rate? • What can we learn from other experiences (ASER/UWEZO/Other)? • What do you think of the suggested approaches? • Any other alternatives to be considered?