Multimodal literacy learning with interactive technologies

1. Multimodal literacy learning with interactive technologies Sue Nichols Centre for Research in Education

2. Multimodality The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning-making resources eg print language, sound & visual images. Some resources used by text producers:

3. Multimodality The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning-making resources eg print language, sound & visual images. Some resources used by text producers: Graphic symbols Spoken language Sound effects Moving images Hyper-linking Concrete materials Written language

4. Sally (aged 4) uses visualresources CONTEXT: Sally had noticed that a researcher visiting her kindergarten was writing in a notebook. She asked to draw something in the notebook. A yoghurt container full of coloured feathers was on the table. Sally announced she would draw that and asked if the researcher would draw the container first. Then the following conversation happened …

5. Sally uses spokenlanguage and social resources Sally: Sally has green feathers and red feathers and purple feathers and yellow feathers and purple flowers. You can write it in your story now. Sue: OK. Sally: I’m going to draw a feather. I’ll do the purple one. You do the next one. Sue: OK. I’ll do the green one. Sally: You do it like that. These are the fluffy bits. You have to go down to here. You have to be careful with textas. You’re doing it nicely, aren’t you? Sue: How’s that? Sally: Good. I’ll do the next one.

6. Sally employs written language resourcesvia an expert helper. Here, the adult helper is the resource, or indeed the technology, which is utilised by Sally to produce the written part of her text.

7. Multimodality is not new • However, digital technologies offer a different set of resources for producing and consuming multimodal texts. • They can make it easier to work across modes and to include modes that have become neglected.

8. Digital word processing has been changing the nature and pace of children’s writing development One of the children had actually sent e-mail messages in September with spaces between words, but did not begin to consistently add spaces in her handwriting for 18 months. • One child typed his name, “Jeffery”, highlighted it, selected the WingDing font type, and gleefully exclaimed, “I made my name in code.” The typed name “Jeffery” had become jefferry Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours. IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta

9. Digital technologies can create different opportunities for children’s communication. • New audiences • New forms of written communication • New kinds of collaboration • Students becoming ‘prosumers’

10. Collaborative online writing In the United States, Common Core State Standards (CCSS) requires teachers to “engage students in collaboration and online publication of writing”. Teachers Mary Blow and Ronica Lawrence, working in schools over 100k apart, decided to partner their classes in an online science research project. http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writing-and-online-publishing accessed 22/8/12

11. Online collaboration using Knowledge Forum Children could see the photographs and the ideas posted by their peers. By clicking on a peer’s entry, text scaffolds in the database would appear that gave children choices; for example: “my theory is…” or “I want to know.” Many children were motivated to make entries themselves using whatever strategies they had available, including use of invented spelling. Pelletier et al 2012

12. Integrating reading & writing with digital devices Amy ( aged 7) & Winnie (aged 8) made notes on their Kindles while reading Friendship According to Humphrey . Amy made 43 notes and Winnie made 33. don’t worry Humphrey i have a terrible life with my sister that is werd what does that line mene i wonder what garths house looked like Sourced from Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher 64(1) pp 15 – 22.

13. Literacy Apps – Blending old and new www.curiouspuppy.com This app is a version of a class children’s book. It is not unusual to see older children’s literature appearing in app form, as it is cheaper for producers than employing authors to create original content. Given what Susan Hill’s research is revealing about the popularity of mobile phones with very young children, this offers a way to engage them with quality children’s literature at little cost to parents. First Published 1979

14. What has changed? • How many of these functions could be carried out without digital technology? • There are significant continuities in so-called ‘New Literacy’ texts and practices.

15. Personal Examples • E-book making in libraries • Photo studio in classrooms In the following slides I share some examples of multimodal texts produced by children in two projects I have been involved in.

16. E-book Making in Libraries • Sessions for children and carers were run in school holiday programs at two metropolitan libraries. • Separate sessions were run for younger (aged 3 – 6) and older (aged 7 – 10) children. • Sessions were 90 minutes long and utilised the iPad app ‘My Story’. • Sessions began with a ‘hands off’ pre-production phase. PRE-PRODUCTION: • Icon recognition (It is vital for users to understand how to access functions like ‘undo.’) • Explanation of multiple modes & non-linear production (Users can commence creating their story with drawing, photography, writing or oral narration.) • Narrative planning ( A ‘hands-off’ time for children and their carers to decide on characters and story line. Deciding on a title is a good focusing move.)

17. PRODUCING THE E-BOOK RESOURCES: • iPads • paper & textas • puppets & figurines • large pieces of coloured cardboard as backdrops • cubes for setting up scenes PROCESS: • Most children began with the visual element. Children chose to either draw or to stage and photograph scenes using props. Some combined both. • The language part of the narrative was either dictated to an adult to type or was recorded as an oral narration by the child. Some children chose to type their own text. • Narratives that had been well planned could be produced more efficiently since children generally moved quickly to the complication part of the story.

18. NARRATED BY CHILD: Once there was a princess

19. NARRATED BY CHILD: Once the princess met a dragon.

20. NARRATED BY CHILD: The dragon ate the princess.

21. NARRATED BY CHILD: He didn’t like her and he spat her out.

22. Created on an iPad by ‘Jack’, a Junior Primary child, using photography, drawing, handwritten text & text typed by an adult but no oral narration. Once there was a fire breathing dragon

23. One day the dragon found that he couldn’t breath fire any more.

24. So the dragon went to see the giraffe.

25. The giraffe waved his magic wand.

26. Suddenly the dragon could breathe fire again. The end.

27. Classroom Photo Studio: Year 1 Children making design choices • For this project, children worked with digital cameras and PowerPoint to produce representations of significant personal objects. • A photo booth was set up in the classroom with a digital camera, tripod, table and display board. Cardboard provided the backdrop. PROCESS: • Children posed and photographed their objects. • The photographs were saved to the class computer. Children then took turns to dictate written text in one or more of three formats: • Speech bubble • Thought bubble • Caption • All the slides were combined in a class presentation that was shown during the school’s Art Night.

28. DESIGN DECISIONS: What object to show; How to pose it; Whether to add text; What text to add; Speech or thought bubble or caption; Colour for text and fill. Eagles are scary. Eagles Aaaah!!

29. Wheeeeee!!! My car knows everything about the world. My car has been all over the world

30. Images of his toy shark taken by a 5 year old boy Adult: Is this how you want it to look? Child: Shakes head Adult: How do you want it to look? Child: Fierce. Image selected by child. Propping shark on plastic cup to show mouth.

31. Questions about any technology • Is it flexible? • Can it be used in combination with existing tools? • Does it foster the integration of skills? • Does it facilitate both scaffolded and independent use by children? • Can it be used for text production as well as for the consumption of texts produced by others? • Does it support different ways of learning and of displaying learning?

32. REFERENCES • Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher 64(1) pp 15 – 22. • Pelletier, J., Reeve, R., & Halewood, C. (2006). Young children's knowledge building and literacy development through Knowledge Forum. Early Education and Development, 17(3), 323 - 346. • Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours.IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta • http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writing-and-online-publishing accessed 22/8/12 TO REFERENCE THIS PRESENTATION: Nichols, S. (2013) Multimodal literacy learning with interactive technologies. Presentation for the Centre for Research in Education, University of South Australia, Adelaide, July 3rd 2013.