Digital Multimodal Writing in ESL Classes

1. Digital Multimodal Writing in ESL Classes Mimi Li, Ph.D. Texas A&M University-Commerce

2. Hot Topic: Multimodal Writing • TESOL Quarterly 2015 Special Issue: Multimodality: Out from margins of English language teaching • Journal of Second Language Writing 2017 Special Issue: Second language writing in the age of computer-mediated communication • JSLW 2017 Disciplinary Dialogues: Perspectives on multimodal composition • JSLW 2019 Special Issue Topic: Multimodal composing in multilingual learning and teaching contexts

3. Background • In 21st-century social and cultural contexts, meanings are more and more represented multimodally(Kress, 2010). • Multimodalities: visual, audio, gestural, spatial, linguistic means of creating meaning (Selfe, 2008) • The paradigm shift in terms of writing representation moves from the logic of the page to the logic of the screen (Kress, 2003), which coincides the visual turn in writing studies: “seeing texts as visual; treating images as texts”(Purdy, 2014, p. 615).

4. CMC technologies have impacted “the ways we write, the genres we create, the authorial identities we assume, the forms of our finished products, and the ways we engage with readers” (Hyland, 2016, p.40).In ESL writing contexts, students draw on digital technologies and are engaged in multimodal writing projects (including blogging, digital storytelling, and video-making), in which they orchestrate various semiotic resources (e.g., texts, images, sound, video, hypertext) to create digital products (Kress, 2010).

5. Rationale • In the digital age, text-based communication cannot adequately support students’ writing in the rhetorical situations they experience. • Digital multimodal writing offers rich opportunities for learners to deploy multiple resources to make meaning, construct knowledge, and express self-identity.

6. Benefits • Fostering learner autonomy(Hafner & Miller, 2011) • Real-world motivation to develop language and literacy skills (Hafner, 2014) • Opening up new identities for L2 learners,enhancing their self-esteem (Lam, 2000), and showcasing the knowledge of their heritage culture and language.(Smith et al., 2017; Vandommele et al., 2017) • Investment change: from resistant writer to active composer; from exam-oriented writer to multimodal designer. (Jiang, 2018) • Positive impact on writing/language development: pronunciation and vocabulary development, increased language production, syntactic complexity, better use of writing conventions (Darrington & Dousay, 2015; Oskoz & Elola, 2016; Yang & Wu, 2012).

7. Representative Cases --- Smith et al. (2017); Bloch (2015); Hafner (2014, 2015)

8. Case 1: Multimodal Presentation-Smith et al. (2017) • Context and participants: Secondary ESL in the US; 8th graders. • Writing task: “My Hero” multimodal project after learning a literature unit connected to The Warrior’s Heart • Conduct/record the interview with their hero. • Create a multimodal presentation that introduces their hero, synthesizes the interview, and includes personal reflections. • Share their work in class as well as in a project showcase at their grade level.

9. Case 1: Smith et al. (2017) Cont’d Sample work: “My mom”: mother’s experience fleeing from EL Salvador Cover page: Six images as a structural blueprint to writing

10. Case 1: Smith et al. (2017) Cont’d • Students were engaged in the multimodal “codemeshing” process (e.g., image design, text revision, voice record, audio remix) and learned to express themselves through multiple modes in digital environments. • Students used the heritage language for different purposes, e.g., searching info, translating to clarify content, engaging with potential readers, and expressing their identity.

11. Case 2: Digital Storytelling-Bloch (2015) • Context/Participants: Tertiary ESL in the US; college students in EAP class • Multimodal writing task: Digital storytelling (DS) • Focused on personal experiences that contained a major turning point in the students’ lives. • Purpose: to help students develop a metacognitive understanding of the choices they make for expressing their voices through different modes.

12. Case 2: Bloch (2015) Cont’d • Example: https://vimeo.com/17709929 • More compelling version of the author’s story than words alone on paper; • A digital story can be simultaneously narrative, exposition, and argument.

13. Case 3: Video Documentary-Hafner (2014) • Context: Tertiary ESP Class in Hong Kong; college students majoring in science • Writing task: Collaborative digital video project- multimodal scientific documentary • Conduct a scientific inquiry (background reading on their topic) • Collect/analyze the data • Script/storyboard • Perform/record • Share the final product on YouTube

14. Case 3: Hafner (2014) Cont’d

15. Case 3: Hafner (2014) Cont’d • Different types of remix were evident: Chunking(mixing sources); layering (mixing modes); blending (mixing genres); intercultural blending (mixing cultural resources). • Students combined a range of modes to develop an effective rhetorical “hook” and appropriate discoursal identity (e.g., scientist, reporter) so as to appeal to their audience.

16. Technology Tools • Glogster (interactive poster) • Aegissub(video subtitling) • WeVideo (video maker) • VoiceThread (video maker) • Powtoon (animated video maker) • Vyond(animated video maker) • Storybird(digital book) • Wix(blogging) • Weebly (blogging) Digital Writing

17. Toolbox • Storybirdhttps://storybird.com/ Artful storytelling (pictures & texts) • Example: https://storybird.com/books/if-i-could-do-anything-52/ • How to create? • Glogsterhttp://edu.glogster.com/ Interactive multimedia poster (pictures, texts, videos, and hyperlinks) • Example: https://edu.glogster.com/glog/womens-role-in-history/2pt3gct3o7f?=glogpedia-source

18. Toolbox • Vyondhttps://www.vyond.com/ Animated video creator • Aegisubbhttp://www.aegisub.org/ Subtitle creator and modifier • How to do?

19. Pedagogical Recommendation --- Task design and Assessment

20. Task Design • Writing to learn: Connected to content areas • Authentic tasks • Develop a travel brochure (Glogster) • Publish a newspaper article (Wix) • Create an E-book (Storybird) • Create a video on science report (WeVideo)

21. Digital Documentary Digital Storytelling Fansubbing New Writing Genres Amateur subtitling of TV shows, movies, and anime(Perez-Gonzalez, 2007). It supports students’ linguistic development and enhances their cultural knowledge and awareness. (Sauro, 2017) Students search and evaluate online information, remix online texts, and create multimodal texts to inform audience on a special topic of their disciplines. (Hafner & Miller, 2011) Students write the script, find images, select background music, and narrate the story while producing a video (Godwin-Jones, 2012).

22. Adsanatham (2012) Li & Storch (2017) Hung, Chiu, & Yeh (2013) Assessment Assess not only traditional aspects of writing, but also aesthetics of design including coordination of texts, images, and sounds Develop a rubric guided by the five design modes: linguistic, visual, gestural, auditory, and spatial (New London Group, 2000) Link assessment to instruction by inviting students to generate grading criteria before composing, which scaffolds composing process

23. References Adsanatham, C. (2012). Integrating Assessment and Instruction: Using Student-Generated Grading Criteria to Evaluate Multimodal Digital Projects. Computers and Composition, 29(2), 152-174. Belcher, D. (2017). On becoming facilitators of multimodal composing and digital design. Journal of Second Language Writing,38, 80-85. Bloch, J. (2015). Digital storytelling: Developing academic writing skills through multimodal texts. In M. Roberge, K. M. Losey, & M. Wald (Eds.),Teaching US-educated multilingual writers (pp. 178–204). Ann Arbor MI: University of Michigan Press. Darrington, B. & Dousay, T. (2015). Using multimodal writing to motivate struggling students to write. TechTrends, 59(6), 29-34. Godwin-Jones, R. (2012). Digital video revisited: storytelling, conferencing, remixing. Language Learning & Technology,16(1), 1-9. Hyland, K. (2016). Teaching and researching second language writing (3rd ed.). New York, NY: Routledge. Hung, H., Chiu,Y., & Yeh, H. (2013). Multimodal assessment of and for learning: A theory-driven design rubric. British Journal of Educational Technology, 44(3), 400-409. Hafner, C. (2015). Remix culture and English language teaching: The expression of learner voice in digital multimodal compositions. TESOL Quarterly, 49(3), 486-509. Hafner, C. A. (2014). Embedding digital literacies in English language teaching: Students’ digital video projects as multimodal ensembles. TESOL Quarterly, 48, 655–685. Hafner, C.A. & Miller, L. (2011). Fostering Learner Autonomy in English for Science: A collaborative digital video project in a technological learning environment. Language Learning & Technology, 15(3), 201-223.

24. References (cont’d) Kress, G. (2003). Literacy in the new media age. London, England: Routledge. Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. New York. NY: Routledge. Li, M., & Storch, N. (2017). Second language writing in the age of CMC: Affordances, multimodality, and collaboration. Journal of Second Language Writing, 36, 1-5. Purdy, J. P. (2014). What can design thinking offer writing studies? College Composition and Communication, 65,612–641. Jiang, (2018). Digital multimodal composing and investment change in learners' writing in English as a foreign language. Journal of Second Language Writing, 40, 60-72. Lam, W.S.E. (2000). L2 literacy and the design of the self: A study of a teenager writing on the internet. TESOL Quarterly, 34(3), 457-482. McDermott, M. (2010). Using multimodal writing tasks in science classrooms. The Science Teacher, 32-36. Oskoz, A., & Elola, I. (2016). Digital stories in L2 education: Overview. CALICO Journal, 33(2), 157-173. Smith, B. E., Pacheco, M. B., & de Almeida, C. R. (2017). Multimodal codemeshing: Bilingual adolescents’ processes composing across modes and languages. Journal of Second Language Writing, 36, 6–22. Sauro, S. (2017). Online fan practices and CALL. CALICO Journal, 34(2), 131-146. Vandommele, G., Van den Branden, K., Van Gorp, K., & De Maeyer, S. (2017). In-school and out-of-school multimodal writing as an L2 writing resource for beginner learners of Dutch. Journal of Second Language Writing, 36, 23–36. Yang, Y., & Wu, W. (2012). Digital storytelling for enhancing student academic achievement, critical thinking, and learning motivation: A year-long experimental study. Computers & Education, 59, 339-352.

25. Contact Dr. Mimi Li at mimi.li@tamuc.edu