UNICEF/WG MODULE ON CHILD FUNCTIONING: TESTING and ANALYSIS

1. UNICEF/WG MODULE ON CHILD FUNCTIONING: TESTING and ANALYSIS

2. Content and structure • Questions ask about difficulties the child may have in doing certain activities • Questionnaires for children 2-4 and 5-17 • Response categories (with the exception of behavior (2-4) and emotions (5-17) are: • No difficulty • Some difficulty • A lot of difficulty • Cannot do at all

3. Domains of Functioning • Seeing2-17 years • Hearing5-17 years • Mobility2- 4 years • Self-care • Fine motor • Communication/Comprehension • Learning • Remembering • Emotions: anxiety and depression • Controlling behaviour • Focusing attention and concentrating • Coping with change • Relationships • Playing

4. Cognitive Testing • Cognitive testing • January 2013, Belize • April 2013, Oman • July 2013, Montenegro • 2012/13/14/15/16, USA • March 2016, India • April 2016, Jamaica • Comparative report completed and decisions made on final set of questions included in field testing

5. Cognitive testing of the child functioning module • Child disability questions perform differently than adult disability questions due to: • Parent proxy • Parent’s knowledge of “what is normal” for children of the same age • Relationship between parent and child • Parental frustration with child • Potential for false positives. • Domains most sensitive to false positives : those that generate parent frustration (learning/remembering, communication, attention/completing a task)

6. Cognitive testing of the child functioning module • Relevance of comparison with other children of the same age: respondents do not always make this comparison/such comparison is not always possible. • Domains most likely to be effected: Self-care, Emotions, Attention • Responses “some difficulty” may lead to false positives • Questions were modified to reduce risks of bias, clarify concepts and facilitate interview process

7. Example Hearing Domain: Round #1 • Does [name] have difficulty hearing? • This question is intended to focus on auditory hearing. • Many respondents, however, focused on listening. Round #2 • Does [name]have difficulty hearing sounds like peoples’ voices or music?

8. Example of possible constructs Compared with children of the same age, does [name] have difficulty with self-care such as feeding or dressing him/herself? Visual Representation of Thematic Schema

9. Field Testing • Independent field testing on earlier version of the module completed in Haiti (Brown University, 2013), Cameroon & India (London School of Hygiene and Tropical Medicine, 2013), and Italy (NSO, 2013) • Field testing of complete version of the module in Samoa (NSO, 2014) and El Salvador (NSO, 2015) with technical assistance from UNICEF/WG • Module also used in surveys in Zambia (National Disability Survey, 2014) and Mexico (MICS, 2016) • Dedicated methodological work in Serbia (NSO, 2016)

10. Field test in Serbia • Primary goal of the field test was to assess the performance of the Child Functioning module in the field • Secondary goal was to compare: • the 2-4 year old questionnaire of the Child Functioning module with the 2-4 year old questionnaire of the Ten Questions (TQ) module • the 5-17 year old questionnaire of the Child Functioning module with the 5-17 year old questionnaire of the Washington Group short set of questions.

11. Feedback from interviewers Questions to be filled out by interviewers at the time of the survey, right after the corresponding questions asked • whether the interviewer had to repeat the question • whether the respondent asked for clarifications of words or concepts • whether the respondent had difficulties with response categories

12. Findings • Questionnaire administered without any major problems • Reactions of the respondents were mostly neutral to positive • Repetitive to read out loud response categories • Developed recommendations on implementation: one randomly selected child per household in countries with high levels of fertility • No need to repeat responses categories after first set of questions unless needed

13. Cut-offs (recommended for international comparisons) • For questions that use the standard WG response options: no difficulty, some difficulty, a lot of difficulty and cannot do at all – cut-off for inclusion is set at a lot of difficulty or cannot do at all • For controlling behavior (2-4) cut-off for inclusion is set at a lot more • For emotions (5-17) the most severe cut-off (daily) is used • Overall measure: % of children with functional difficulties in at least one domain

14. Analysis of findings from testing • Within range disability prevalence obtained across the different settings • Similar patterns in terms of characteristics of the population of children with functional disability • Most common functional difficulties across contexts: controlling behavior, managing emotions, learning

15. Prevalence rates by different cut-offs Samoa, 2015 / Serbia, 2016

16. Disability bySchool attendance Samoa, 2015

17. Results across modules Serbia, 2016

18. Conclusions • Questions carefully drafted to reflect multitude of domains and reduce risk of missing out children with different functional difficulties • Domains are not meant to be looked at in isolation • Questions carefully tested and cut-offs selected to reduce risk of false positives • Not recommended to reduce number of questions, change response categories, modify content or change cut-offs