Terry Inglese UCSB, 14th November 2006

1. Learning with audio-visible and invisible authorsLessons learnt from a Political science course at USI – University of Lugano, Switzerland Terry Inglese UCSB, 14th November 2006

2. My background • TV program designer and archivist at the Swiss public TV • Indipendent documentary filmmaker • PhD student • Multimedia instructional designer Combination of academic and professional skills

3. The memory institutions and the digital revolution • Archives - from storing objects to the management of the life cycle of digital and digitized products; • Libraries - from the reading room to the digital information service centre; • Museums - from collections to narrative connections and new experiences. • from a static into a dynamic change

5. Database and Narrative (Manovich – 2001)

6. “The emergence of new media coincides … with accessing and reusing existing media objects as with creating new ones.” • (Manovich, 2001, pp. 35, 36)

7. TV archive and new media • New media tend to repackage old media through reproduction, manipulation, convergence. • Benjamin (1977) and Baudrillard (1996) • the loss of aura that the original work of art would lose if inserted into the age of mechanical reproduction. • In our case studies, besides an apparent feeling of loss, there is a sense of gain that new media are providing, a gain that is not material, but cognitive and affective.

8. The auratic value of new media • New media are not just invented to meet the needs that already exist, but also new ones. • Kress and Van Leeuwen, “what is lost may return, and what is gained many yet turn out to be loss. The new technologies’ emphasis on multimodality, three dimensionality and interactivity can be seen as a return of many of the things that were lost in the transition from orality to literacy, as a second orality, in other words.” (Kress, Van Leeuwen, 2001, p. 92). • New media possess an auratic value, as the one of the medieval manuscripts, because these media become manuscript media, stored, transmitted, encoded and re-encoded in a digital form.

9. The conceptual orality • From this perspective, archives provide a return to a conceptual orality, that is to say, “a return to the medieval framework wherein words or documents gained meaning only as they were closely related to their context and to actions arising from that context. In that oral tradition, meaning lies not in the records themselves, but (in) the transactions and customs to which they borrow witness as ‘evidences.’ (...).” • (Taylor, in Cook, Dodds, 2003, p. 23).

10. Taylor • “a catalogue remains a catalogue, but pattern recognition is the chef-d’oeuvre of human intelligence”. The added value: • Discovering patterns of knowledge • Extracting more from less (Taylor, in Cook, Dodds, 2003)

11. My work Goal: • to recognize the pedagogical as well as instructional benefits of studying with TV multimedia archives as learning tools, taking into account the students’ perspective. • The genre: TV interviews with Social Science scholars usually read and studied only through texts.

12. RTSI – Swiss public TV and radio • Radio – RSI - 30ies • TV – RTSI - 60ies • because of the lack of the higher education institution like a university, the RTSI produced highly cultural programs (TV and radio) enhancing education and culture (Inglese, 1998) • Genre of TV program – the interview

13. Availability and Affordances concepts availability concept: archival content, produced in the past with analogue technologies, is easily ‘transformed’ into digital support. This technological shift increases the potential availability of ‘multimedia contents’; affordance concept: students learn more deeply when they feel a personal relationship with the author to be studied, if the author (hypothesis) is offered – beside the text - through a multimedia & multimodal format such as a TV interview. (Inglese, Mayer, Rigotti (2007, in press))

14. Media for learning • Media can be defined by: 1. the technology 2. the symbol systems 3. the processing capabilities

15. media attributes and instructional methods • it is not possible to separate the media attributes from the instructional methods • “What we have learned from all the media comparison research is that it’s not the medium, but rather the instructional methods that cause learning.” (Mayer, Clark, 2003, p. 21) • Learning depends on the quality of instructional messages and instructional methods rather than the medium per se.

16. Multimedia learning - Mayer • The cognitive theory of multimedia learning. (2001, 2003, 2005, etc.) • Multimedia: • presentation of material using both words and pictures. • Multimedia learning is “a sense-making activity in which the learner seeks to build a coherent mental representation from the presented material.” • “the learner’s job is to make sense of the presented material; thus, the learner is an active sense maker who experiences a multimedia presentation and tries to organize and integrate the presented material into a coherent mental representation.” (Mayer, 2001, p. 13-15) • The cognitive theory of multimedia learning is consistent with empirical research results based on how people learn and the understanding of the main cognitive processes in learning.

17. Multimedia learning

18. Social agency theory - Mayer • assumptions that social cues in multimedia instructional messages prime a social response in learners • the learner can interpret a multimedia instructional message as • an instance of information delivery or/and • a social communication event. Influencing the type of schemas activated in the learner the type of cognitive processing, and the quality of the learning outcomes • Multimedia principle (Mayer, 2001) • Personalization principle (Mayer, 2003) – I-you vs. 3rd person • Sound cues – voice and the concept of persona (Mayer, 2005, 2006)

19. ‘the media equation paradigm’. • people apply the same dynamics from human-human interactions to human-computer interactions. • Reeves and Nass (1996) define this phenomenon ‘the media equation paradigm’.This paradigm states that the media equal real life. • All people automatically and unconsciously respond socially to media, and therefore also to computer and television, … especially nowadays with the new and old media convergence. • Media features: closeness, size, visual and audio fidelity, motion, memory, and voice.

20. Research questions • Would students learn (read and write) better if teachers could use texts and audiovisual materials, such as TV archived interviews with scholars that are normally studied through texts only? • retention and motivation (did students retain and enjoy more with an audio-visible author vs. an invisible text-based author?) • Grade • Number of written words • Quote • Does reading text of scholars offered through multimedia instructional messages change? If yes, how and why? • the social presence of the author

21. Kamin, Intrator and Kim (2000) have shown despite the appeal of using multimedia documents to enhance reading, there are still several cognitive and affective processes in reading and processing texts that need to be investigated.

22. The case study - Definitions • Audio-visible author • Multimedia and multimodal interview format • I-you relationship • & Text • Invisible author • Only text • No I-you relationship • Detached, formal and academic language (strong text)

23. 1. Case study

24. The final written exam texts • 105 students took the written exam • 73 native Italian speakers and 32 non-native ones. • The non-native Italian students could write it in their own languages: German, French, Spanish or English. • 7 questions: • 2 questions related to the audio-visible authors and • 1 related to the invisible author. • 10-to-1 grading scale, with 10 as the highest grade and 1 as the lowest.

25. Karl Popper – audio-visible author + text • Paul Feyerabend – audio-visible + text • Andrea Semprini – invisible author – only text (a text that is invisible)

26. grades

27. words

28. quotes

29. 2. Case study • 108 university freshmen • 60 native Italian speakers and 48 non-native ones (15 different countries and speaking a total of 13 different languages). • the average age in the class was from 20 to 25 years old

30. Claude Lévi-Strauss – audio-visible author + text • Paul Feyerabend – audio-visible author + text • Andrea Semprini – invisible author

31. grades

32. words

33. quotes

34. Qualitative data • thinking and feeling aloud methodology to collect the ongoing cognitive and affective responses and processes of students while reading the texts aloud. • After reading, three questions: • 1. which is the most comprehensible author and why? • 2. which is the most interesting author and why? • 3. which is the most closer author and why? • 1. to imagine how the invisible author looks like • 2. Bus scenario • 3. Taxi scenario

35. Measuring the author‘s presence in their texts

36. Taxi

38. In deep-interviews with students • Authors • Texts • Reading act • Context

39. Students‘ metaphors • The invisible author: realistic, precise, scolastic, serious, authoritative, rigorous, clear, scientific, vague, hostile, sterile, as an anchor, it does not touch • Reading the invisible author’s texts: heavy, dense, rigid, tiring, hard, twisted, muddled, cold, abstract, complex, distant

40. Students‘ metaphors • The audio-visible authors: sense of movement, participation, feeling of nearness and familiarity, involvement, relationship, access, contact, possibility of identification with the author, clearity, fresh air, sympathy, an encountering that leaves a trace, convincing effect, influencing effect, openness, sense of freedom, an embrace, a testimony, “having more than a weapon (to study)” (italian expression), a breathing, an encountering that touches • Reading the audio-visible authors’s texts: light, fluid, a flowing and pleasant experience, linear, untied, warm, concrete, easy, near

41. 3.rd case study M. Foucault – audio-visible author • (difficult to read and study) B. Constant – invisible (only text) but visible (I-you) author • (easy to read and study)

42. grades

43. words

44. quotes

45. Additional audio-visible authors?

46. Some conclusions • 2 language and multimedia • (Inglese, Mayer, Rigotti, 2007, in press) • Literacy and digital media • Importance of author‘s awareness and the social presence of the author in expository texts • Designing instructional messages more speech-like • Reconsidering Orality in combination with Writing • Importance of using archives for educational purposes, for example TV and radio ones

47. Some interpretations • For Brandt (1990), reading is considered as an involvement act, where readers try to reach across texts to other human beings, having to be more consciously aware of what is taking place on the other side of the communication, as is necessary when the discourse is oral. • Readers read not to separate from others, but to reach out to them. The motive for reading is to find other minds.

48. Some interpretations • Literacy failures are not failures of separation but rather failures of involvement. They arise not from overdependence on context but from the lack of access to a context for making sense of print. • Instead of viewing the oral as antagonist to the literature, it is necessary to understand better how the oral sustains the literate. • (Brandt, 1990, p. 7)

49. Looking for UCSB collaboration • accessing US grant money and US scholars that would like to use these audio-visible authors too, for instructional purposes and for research! • developing together the Social Science audio-visible authors‘ database in collaboration with the RTSI Swiss public TV and USI Lugano.