Examining Early Child Development in Low-Income Countries: A Toolkit for the Assessment of Children in the First Five Ye

1. Examining Early Child Development in Low-Income Countries: A Toolkit for the Assessment of Children in the First Five Years of Life Lia Fernald, Ph.D. Patricia Kariger, Ph.D Patrice Engle, Ph.D Abbie Raikes, Ph.D.

2. Acknowledgements • Inspiration & funding from the World Bank • Barbara Bruns, Sophie Naudeau, Harold Alderman, Ariel Fitzbein • External reviewers • Frances Aboud, McGill University • Santiago Cueto, Catholic University, Peru • Ed Frongillo, University of South Carolina • Jane Kvalsvig, University of Kwa-Zulu Natal, South Africa • Ann Weber, University of California, Berkeley • Paul Wassenich, University of California, Berkeley • Michelle Neuman, The World Bank • Mary Eming Young, The World Bank • Collaborators • Emanuela Galasso, The World Bank • Lisy Ratsifandrihamanana, Madagascar • Lourdes Schnaas, Mexican Institute of Perinatology • Research assistants • Robin Dean (UC Berkeley), Kallista Bley (UC Berkeley), Melissa Hidrobo (UC Berkeley), Anna Moore (Cal Poly) • Photo credits for photographs included in presentation • Lia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber, Tricia Kariger

3. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

4. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

5. Introduction: Why measure child development? >200 million disadvantaged children worldwide Percentage of disadvantaged children under 5 years old by country in 2004 Grantham-McGregor et al., Lancet (2007)

6. Ecological model of child development Adapted representation of Bronfenbrenner’s ecological model of child development (Wortham, 2007)

7. Direct & Indirect Effects Conceptual framework • Environmental factors • Psychosocial risks: harsh disciplinary techniques, maternal depression • Biological risks: malnutrition and infectious diseases • Poverty and socio-cultural factors increase likelihood of both types of risks From Walker and al. Lancet, 2007

8. Timeline of development • Early childhood is characterized by developmental spurts and plateaus • Skills emerge at different rates and ages Timing of human brain development, from Grantham-McGregor, et al., 2007

9. Differential risk and vulnerability • Children’s development from 0-5 is dependent on quality of early environments and relationship with caregiver. • Young children growing up in poverty are disproportionately exposed to a wide range of risk factors: • Poor nutrition • Less stimulating learning environments • Poor sanitation • Stressful life events • Exposure to environmental risks

10. Poverty and cumulative risk • Number of risk factors increases over time. • Cumulative effect of risk factors becomes more evident as children get older • Higher cumulative levels of risk are associated with: • Poorer cognitive development • Psychological distress and behavioral problems • Slower and lower quality communicative development

11. Cultural norms and development • Cultures have a wide range of values for when and how skills and abilities develop in children. • As school becomes more universal, however, the necessary skills become more consistent across cultures. • Through modification and adaptation, every effort must be made to ensure that tests are fair for all children assessed.

12. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

13. Domains of development • Domains of development: • Cognitive • Language • Motor • Executive function/self-regulatory • Social/emotional • Domains are overlapping and mutually influencing • Every effort should be made to include multiple domains when assessing children’s development

14. Cognitive skills • Cognitive skills include: Analytical skills, mental problem-solving, memory, and early mathematical abilities • Indicators: • Children near school age: knowledge of letters and numbers, ability to retain information in short term memory, knowledge of key personal information • Children in school: knowledge of letters and numbers, reasoning, problem-solving, memory, and mathematical abilities

15. “DAY” “NIGHT” Executive function • Defined as fluid abilities or processes that are engaged when a person is confronted with a novel situation, problem or stimulus • Both cognitive and emotional processes are involved • Cognitive: remembering arbitrary rules and other non-emotional aspects of the task • Emotional: inhibition or delayed gratification • Indicators: • Working memory • Inhibition of behavior or responses as demanded by the task (e.g. not opening a box until a bell rings) • Sustaining attention as required or ability to switch attention as necessary (e.g. Shifting focus from the color of a test stimulus to the shape of the stimulus)

16. Language development • Early indicators (infancy): babbling, pointing, and gesturing. Use maternal report during this period. • Later indicators (preschool years): production and understanding of words, ability to tell stories, identify letters, comfort and familiarity with books. Can use direct assessment. • Quality and speed of development highly dependent on quality of caregiving environment

17. Motor skills • Large motor: acquisition of movements that promote an individual’s mobility (useful to measure in young children) • Contributing factors: brain and neuromuscular maturation, physical growth, caregiving practices, opportunities to practice emerging skills • Fine motor: involves hand eye coordination and muscle control (e.g. drawing, holding utensils, etc.) (more relevant for older children)

18. Socio-emotional development • First two years: relationships with caregivers, attachment, trust, and early strategies for dealing with negative feelings • Preschool years: social competence, behavior management, social perception, self-regulatory abilities

19. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

20. Step 1: Define purpose of assessment

21. Key Questions in Selecting Instruments • What are the goalsof the assessment/evaluation? • What dimensions of child’s development do you expect to be affected by the intervention? • What developmental systems are most vulnerable at a given age range? • What are immediate outcomes and longer term outcomes? • What are the mechanisms at work? • What physiologic processes are influenced by iodine/iron/poverty? • What are key elements of contextthat must be considered in selecting the test? • Urban/rural, level of poverty, parent education. • At what level will effect be measured? • Individual? Household? Population (then consider test such as EDI)? • How will the sample be selected? • Population sample? Sub-sample? • What is the analytic plan? • Are norms relevant and/or available? Will a cut-off score be used?

22. Step 2: Determine type of assessment

23. Screening versus ability test • Screening tests: brief assessments to identify children who are at risk of having development problems • Inexpensive, quick, and relatively easy to administer • Classify children into categories • Cutoffs used in one population to classify children should not be applied to another population! • Ability tests: longer tests that assess the maximum skill level for a child at any given age • Continuous scores that can be used to compare children’s developmental levels with more precision

24. Step 3: Determine mode of assessment

25. Types of assessments: Direct tests • Pros: • Data are gathered first hand • Data can be less biased than parental reports • Potentially wider range of outcomes can be assessed • Many of the “cons” can be overcome with careful planning and preparation • Cons: • Young children can be difficult to test (sleeping, hungry) • Testers need a lot of training and oversight • Accuracy depends on testing demands and child must be familiar with parameters (e.g. best v. worst)

26. Types of assessments: Parent report • Pros • Easy to administer and require minimal training and instruction • Often are quick and easy to complete and to score • Parents can become involved and express concerns • Often correlate well with direct assessments • Teachers can be an additional source of information as children get older • Cons • Parents and teachers may artificially inflate scores • Parents may not accurately report abilities • Parents and teachers may have different interpretations of items in different cultures

27. Types of assessments: Observation Types of observation: Naturalistic observation, Sampled observation, Structured situation • Pros: • Highly valid • Measures behavior in an identified context • Can provide additional or confirmatory information for other types of assessments • Cons: • Requires a lot of time and training • Need to identify if culturally appropriate • Difficult coding since observational codes and definitions are not always clearly defined

28. Step 4: Determine which assessment to use

29. Other constraints to consider • Budget: Tests can be very expensive (e.g. $1000 for Bayley); administration time is a budget issue, too. • Copyright issues: Must obtain permission for most tests. • Time allocated for testing: Direct assessment v. parent rep. • Training: Capacity for administration. • Test setting: Set-up, lighting, noise, observers • Capacity of respondent: Education/knowledge of parent • Language and cultural differences: Words used in testing materials, approach used for testing (e.g. speedy response) • Materials: Must be familiar and/or available (e.g. mirror, ball)

30. Ethical risks and responsibilities • All assessment protocols must be reviewed and approved by an ethical review board • Accuracy and validity are extremely important especially if test scores are being used to identify children “with delays” • Follow-up (e.g. referrals for at-risk children) should be mandatory even in the context of a developing country.

31. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

32. Instruments: Modifying and adapting • No test is “culture free” • Construct bias (e.g. test doesn’t measure “intelligence” the same way in both cultures) • Method bias (e.g. procedures are unfamiliar and differentially affect responses) • Item bias (e.g. individual test items do not translate well) • Existing tests that are reliable and valid can be used across different cultures but they must be modified and adapted to achieve: • Linguistic equivalence • Functional equivalence • Cultural equivalence • Metric equivalence (level of difficulty)

33. Preparatory work for test adaptation • Involve local professionals to gather information relating to linguistic, cultural and technical details that could be relevant. • Psychologists, community health workers • Early childhood educators • Produce an accurate translation • Translation and back-translation • Review, comparison, correction • Pilot translated version to explore possible areas of confusion

34. Steps for test adaptation • Adapt test content to local context • Make as many changes as necessary while maintaining the intended “meaning” of the item • Examples

35. Example: Modifying Peabody test Change from dollars to Ariary Remove stairs from bannister

36. Example: Modifying Peabody test Replace skiing child with sledding/skating child Modify tractor, smaller

37. Example: Modifying Stanford Binet • Most materials could be used as intended • Description of picture included automatic washing machine – changed to traditional wash board

38. Example: Modifying Leiter test Replace ram with pig Replace car with tractor Replace flag Replace straight hair with curly Replace reindeer with ox

39. Example: Modifying ASQ • When in front of a large mirror, does your baby smile or coo at herself?

40. Example: Modifying ASQ • When in front of a large mirror, does your baby reach out to pat the mirror?

41. Example: Modifying Motor tests

42. Steps for test adaptation, cont’d • Adapt administration procedures • Tester (e.g. affect, responsivity, sensitivity, development of rapport, willingness to change environment) • Test environment (e.g. materials, table, chair, lighting, sound, observers, other distractions) • Test procedures (e.g. accuracy of parent response, clarity of instructions)

43. Example: Modifying ASQ Will caregivers make accurate assessments of their children’s development? • We added 5 demonstration items to • Provide children a chance to demonstrate behaviors that may not be easily observed (looking at pictures in a book; looking in a mirror) • Act as a validity check of parent responses

44. Example: Modifying ASQ Without showing him first, does your child point to the correct picture when you say, “Show me the kitty” or ask, “Where is thedog?” GIVE THE PICTURE TO THE CAREGIVER AND ASK HER TO SHOW IT TO HER CHILD.SAY TO THE CAREGIVER: “I know children do not always do what they are asked, but let’s see if he will do this for us today. Go ahead and ask [CHILD] to show the kitty, dog, ball or shoes.” INSTRUCT THE CAREGIVER NOT TO POINT TO ANY PICTURES. YOU CAN ALLOW ABOUT ONE MINUTE FOR THE CHILD TO DEMONSTRATE THE BEHAVIOR.

45. Example: Modifying ASQ Can we adapt the majority of items across all cultures? • We added clarifications where items seemed ambiguous • Does your baby get into a crawling position by getting up on her hands and knees? [BABY DOES NOT HAVE TO CRAWL, BUT MUST BE ABLE TO MAINTAIN SELF ON HANDS AND KNEES.] • Does your child drink without help from a cup or glass, putting it down again with little spilling? [CHILD CAN DRINK ALONE FROM A CUP WITHOUT SPILLING TOO MUCH.]

46. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

47. Requirements for creating a new test • Involve an inter-disciplinary research team • Use a representative sample for testing items and test cohesion • Conduct a detailed analysis of the instrument’s psychometric properties • Develop norms or standards that represent typical development in the population under study

48. Examples of new tests • Africa • Kilifi Developmental Inventory -assesses psychomotor development in a resource-limited setting • Grover-Counter Scale of Cognitive Development -developed in South Africa to assess the level of cognitive functioning of children 3-10 years with impaired verbal skills • Asia • Cambodian Development Assessment Test – measures level of cognitive, social, motor, and academic development based on country specific standards • Latin America • Test de Desarollo Psicomotora -developed in Chile, it evaluates child development in motor function, coordination, and language • Escala de Evaluacion del Desarrollo Psicomotor – screening measure of language, social, coordination, and gross motor skills. Norms and cutoffs developed for Chile.

49. How to develop a set of Standards Define domains Within each domain, define a set of standards or goals For each standard, outline the specific objectives and indicators for each age level Pros of Standards approach Culturally appropriate Process can be informative Cons of Standards approach Time-intensive and requires long term follow-up Indicators are not necessarily translated into a test Needs to be done slowly and carefully Example from Vietnam for children 5-6 years old Using the “Standards” approach NOTE: UNICEF has worked with over 40 countries to develop Standards

50. Today • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations