Localized Iterative Design for Language Learning in Underdeveloped Regions: The PACE Framework

1. Localized Iterative Design for Language Learning in Underdeveloped Regions:The PACE Framework Matthew Kam Divya Ramachandran Varun Devanathan John Canny Berkeley Institute of Design & Computer Science Division University of California, Berkeley Anuj Tewari Dhirubhai Ambani Institute of Information & Communication Technology (India) CHI 2007San Jose, CA

2. Relevance of English as a Second Language (ESL) in India • English is a global language: 1.2 to 1.5 billion people in >170 countries (Crystal 1997) • ESL is taught in almost all schools in India • Mastery of English is the “single most influential factor that determines access to … important avenues of economic and social advancement” (Kishwar 2005) • 90% of indigenous web content in India is in English

3. “World Language” Fluency • Low-income populations in Africa, Asia and Latin America desire to improve command of an appropriate “world language” (Clegg, Ogange & Rodseth 2003; Faust & Nagar 2001; Kapadia 2005) • Widely spoken language, e.g. English, Mandarin, Spanish, etc. • Regional dialect is not necessarily native language for low-income groups • “World language” fluency opens the door to further education, “New Economy” jobs, higher incomes, social prestige, etc.

4. Schools Fail at “World Languages” • Schools in developing countries have limited impact • For example, in India: • Rural ESL teachers communicated with us through interpreters • 43% to 61% of school-going-age children do not attend school regularly (Azim Premji Foundation 2004)

5. Our Envisioned Solution • Mobile games that target learning anytime, anywhere • Make ESL learning resources more accessible • Make learning process more enjoyable • Run on cellphones, the fastest growing technology platform in the developing world • Target local language learning needs

6. Case for e-Learning Games • Enhance motivation and learning (Jenkins 2005) • Incorporate good learning principles (Gee 2003) • Prior randomized experiment (Banerjee et al. 2005) • 2 years, >10,000 urban slums students in India • Played math computer games twice per week • Significant gains in math test scores

7. Central Question How can we promote reuse when localizing language learning software applications with communities in the developing world? Recurrent challenges: • Limited time for iterative design in the field • Wide disparity in age, etc. for each literacy level • Limited exposure to high technology

8. Related Work: Localization • Adapting software for a local context involves two steps • Internationalization • Localization • Previous work have mostly focused on the user-interface, not content (e.g. Marcus and Gould 2000, Smith et al. 2004, Yeo 2001)

9. Design Pattern • “Template” solution to previous problem • For ESL learning task, a design pattern is the steps that learner engages in to develop language skills • Benefits: • Represent solutions to frequent problems in skeletal form • Encourages reuse, avoids reinventing the wheel

10. Example Learning Activity (from: Rosetta Stone) • Word-picture matching activity • Targets vocabulary building and listening comprehension • Teaches four words • Highlights each picture as its word is played aloud • Then tests learner

11. Example Pattern Pattern name:Oral Word  Semantics Association Focus: vocabulary building, listening comprehension Solution(Suppose X=4, i.e. teach 4 words per round): Repeat step 1 for X times: 1. Says a word and displays its meaning pictorially Repeat steps 2-3 for X times: 2. Displays one of the X words from step 1 3. Give learner at least X pictures to choose from, and feedback on whether or not his choice was correct

12. Related Work: Design Patterns • Patterns have not been used in instructional design for language learning nor software localization • Have been used in • Urban planning and architecture (Alexander 1977) • Software engineering (Gamma et al. 1995 aka “Gang of 4”) • Interaction design (Borchers 2001) • Website design (Van Duyne et al. 2002) • Computer science education (Bergin)

13. PACE Framework • The four components of PACE framework • Pattern • Activity • Curriculum • Exercise • Pattern  Activity  Curriculum  Exercise • Modular design and reuse • Localize only those parts that need changing

14. Early Lessons (Spring 2006) • Worked on over 30 ESL learning games • Introduced patterns after finishing initial designs • Initial lessons • Use patterns as a “generative” design tool, not in “formulaic” manner • Facilitate iteration on user-interfaces and curriculum by keeping them separate Pattern  ?  ?  Exercise on cellphone

15. On to Round 2! (Summer 2006) • Distilled >50 patterns • Shortlisted 11 patterns for language beginners • Letter-sound correspondences (reading and writing skills) • Word-meaning associations (listening and reading skills) • Pronunciation (speaking skills) • Syllable segmentation (reading skills) • Consulted ESL teacher with teaching experience in rural Central America on these patterns PACE: Pattern Activity  Curriculum  Exercise

16. Sample of Commercial Packages • Reviewed >35 language learning applications from commercial market • Sample was selected based on proxy indicators • Professional customer base • Excellent ratings from previous purchasers on e-commerce, home schooling, etc. websites • Selected for balance between listening, reading, speaking and writing skills PACE: Pattern Activity  Curriculum  Exercise

17. Approach to Extracting Patterns • Theory • First language acquisition for children (Bruner 1983) • Second language acquisition for children & adults (Bialystok & Hakuta 1994, Krashen 2003) • Reading acquisition(Snowling & Hulme 2005) • Practice • ESL teaching methods (Krashen & Terrell 1983, Larsen-Freeman 2000) PACE: Pattern Activity  Curriculum  Exercise

18. Learning Activities • The computer-mediated activity and its user-interface • Concrete instantiation of an (abstract) design pattern • Designed 9 learning activities based on the 11 shortlisted patterns • Design goals • Learnability • Fun and engagement (Malone 1980) PACE: Pattern  Activity  Curriculum  Exercise

19. Curriculum • Syllabus that learner is supposed to learn • Alphabet and context-specific vocabulary • Scenarios for school, nature, traveling & shopping • Numbers, dates and time • Images from clipart libraries were edited for cultural appropriateness • Audio voiceovers recorded with Indian accent PACE: Pattern  Activity  Curriculum  Exercise

20. Exercises • Exercise = Activity + Curriculum • Implemented 6 learning activities to be piloted in Uttar Pradesh, India in late Summer 2006 GraphemeGrapheme correspondence GraphemePhoneme correspondence SemanticWord correspondence Syllable segmentation PACE: Pattern  Activity  Curriculum  Exercise

21. Summary: Our PACE Process • Localized highly-rated commercial language learning packages for an underserved community • Pattern  Activity  Curriculum  Exercise word-picturematching oncellphones “Stop” … Pattern name:Oral Word  Semantics Association Focus: vocabulary building, listening comprehension Solution(Suppose X=4, i.e. teach 4 words per round): Repeat step 1 for X times: 1. Says a word and displays its meaning pictorially Repeat steps 2-3 for X times: 2. Displays one of the X words from step 1 3. Give learner at least X pictures to choose from, and feedback on whether or not his choice was correct “Conductor”

22. Iterative Design in the Field • India: afternoon school for urban slums girls • Kindergarten and 1st grade students (14 girls) • ESL baseline: some knowledge of alphabet • Played 6 exercises over 3 days • 6th grade students (11 girls) • ESL baseline: simple sight vocabulary and basic conversational greetings • Played 12 exercises over 5 days • No prior cellphone experience

23. Benefits: Multiple Age Groups • Reused the same curriculum for younger learners • Learning activity for kindergarten children • Learning activity for older children • Learning activity for adults “A”, “E”, “I”, “O”, “U” Cluster of English alphabet (e.g. vowels) ? One curriculum for many activities

24. Benefits: Comprehensive Suite • Reuse the same curriculum for multiple learning needs, after extensive iteration • Learning activity for spelling • Learning activity for listening comprehension • Learning activity for pronunciation practice Vocabulary for travel-related words One curriculum for many activities

25. Benefits: Learnability • Reused the same learning activity for multiple curricula, to promote learnability • Vocabulary for travel-related words • Vocabulary for shopping-related words • Vocabulary for other functional categories Learning activity for word-picture association One activity for many curricula

26. Positive Outcomes: Learning • PACE permitted rapid iterative design in the field • User acceptance issues ironed out in time • Kindergarten and 1st grade students • Most completed 2 or 3 exercises, out of 3 • 6th grade students • Post-test vocabulary gains (p < 0.001, effect size = 1.16, n = 11) • Transfer to story-writing domain

27. Conclusion • Design patterns • Capture best practices (internationalization) • Facilitate adaptation to local contexts (localization) • Yield positive learning outcomes • PACE framework facilitates • Rapid iterative design in the field • Flexibility for differences in age groups and learner levels • Reuse, learnability and scalability

28. Acknowledgement • Local community partners (India) • Urvashi Sahni, Pratim Basu, Siddharth Bhagwani • Colleagues • Ruth Alexander, Asya Grigorieva, Dimas Guardado, Jeffrey Heer, Kristopher Hom, Maksim Lirov, Aaron McKee, Anand Raghavan, Priyanka Reddy, Aretha Samuel, Monish Subherwal, Susan Woolley • Anonymous reviewers and Associate Chairs • National Science Foundation (Grant No. 0326582) • Microsoft Research (Digital Inclusion award) • Intel Undergraduate Research Program