Download
1 / 29
330 likes | 589 Views
Part: 2 Multimediality: from transcriptions to multimodal corpora. The Westpac advertisement: a multimodal analysis and transcription of a dynamic text. Paul Thibault, University of Venice. Now published as Paul J. Thibault (2000) “The multimodal transcription of a
E N D
Part: 2 Multimediality: from transcriptions to multimodal corpora The Westpac advertisement: a multimodal analysis and transcription of a dynamic text Paul Thibault, University of Venice Now published as Paul J. Thibault (2000) “The multimodal transcription of a television advertisement: theory and practice” in Anthony Baldry (ed.) Multimodality and multimediality in the distance learning age, Campobasso: Palladino Editore, pp. XXX-XXX Workshopon multimodality and multimediality in the distance learning age. Birmingham 16.10.2000 Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
The Westpac video (from Australia,1983) Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
The transcription that is presented below is itself a multimodal text. Specifically, it is a Table Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 1: Time Column 1 specifies the time in seconds of the video recording. This was determined by the time indicator in the Windows multimedia reader and its accuracy may be checked simply by sliding the progress bar with the aid of the mouse from one frame to the next. In this way, I believe that a more accurate reading of the correlation between time-per-second and Visual Frame can be obtained than would be the case with the use of a stopwatch used in conjunction with a video playback facility. Column 1 is also important in a second, related way. The numbers in this Column also serve to identify the horizontal Row with which the time specification correlates. Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 2: The visual frame • The Visual Frame correlates with the time that is indicated in the first Column. Each frame was inserted into the Column by copying selected frames of the entire film text from the .tif file that was obtained from the video text. Clearly, the use of so many frames – more than sixty in the present case – is expensive from the point of view of their reproduction in print form. However, there are important analytical advantages to be gained if this approach is followed rigorously. In my experience, the main advantage lies in the discipline and exactitude that it imposes on the entire transcription procedure.. Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 3: The visual image Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 4: Kinesic Action • The fourth Column, which is headed ‘Kinesic Action’, refers to the use of body movements of various kinds. In this Column, I have grouped together a number of different kinds of ‘behavioural units’, as defined by Kendon (1981:1981), or ‘spatiotemporal arrangements’ of the agents in some discursive event. In the Westpac text, salient behavioural or kinesic units include bodily actions such as ‘smiling’, ‘rolling the sleeves up’, ‘gaze’, and ‘moving/walking forward’. It is doubtful that such kinesic units have a fixed or univocal meaning which can be established independently of their cross-modal relations with other features of the text as a whole. Indeed, identical body movements may have quite different significances according to their patterned relations with other features in other (con)texts. Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 4: Sample of kinesic action Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 4: How to interpret kinesic acts? The kinesic act ‘rolling the sleeves up’ may be interpreted as a gestural emblem with the culturally fixed meaning of ‘starting or getting on with the task to hand’. However, it does not follow that this body movement always has this particular cultural meaning. In some other context, the same action may be a purely physiological response to climactic conditions, with no specific semiotic significance in a given interactional context. What, then, helps us to motivate a specific semiotic significance for a given bodily act? Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 4: a sample of kinesic transcription Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 5: The Soundtrack In this Column, speech, music, and other sounds will be brought together on the assumption that they all have characteristics in common which provide a basis for talking about them and transcribing them in a unified way rather than as entirely separate phenomena. The guiding assumption here is that the acoustic flux of the soundtrack is a perceptual continuum constituting a delimited auditory array which, however, listeners are able to analyse or parse into different components of information that tell us about a given source. The soundtrack is delimited rather than ambient because it derives from a specific point source coming from a particular direction – the loudspeakers, say - rather than the ambient auditory array which surrounds us and comes from all directions as we move through and orient to a natural or urban environment. Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 5: Soundtrack Sample Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 5: Sample Transcription of Soundtrack Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 5: Soundtrack Transcription System In order to distinguish music, speech, and other sounds in a suitably retrievable form, the following notational conventions will be adopted: [♫] = instrumental music (e.g. Row 2, Column 5);[♫♀] = female soloist (e.g. Row 17, Column 5);[♫♂] = male soloist (not attested here);[♫♀chorus] = female chorus (e.g. Row 16, Column 5);[♫♂chorus] = male chorus (not attested here); [☻♀] = female speaker (not attested here);[☻♂] = male speaker (e.g. Row 31, Column 5); [±] = source of spoken voice off-screen, not shown in depicted world (e.g. Row 31, Column 5 ff.); Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Column 5: Soundtrack Transcription System [☼sheep] = other non-speech or non-musical sounds, including silence, followed by a brief verbal specification of the specific sound; in the present example, the sun symbol followed by the word ‘sheep’ designates the (non-linguistic/non-musical) sounds of the sheep, as in Row 2, Column 5;[☼silence] = silence other than rhythmic pause or juncture in speech and/or music (e.g. Row 1, Column 5);↓ = continuation of previous, as, for example, when sung or spoken text is stretched over more than one Visual Frame or Shot (e.g. Row 3, Columns 3 and 5). As the example here shows, this notational symbol is not specific to the soundtrack and is also used in other Columns. In all cases, it has the same significance. Workshop on “Multimodality and Multimediality in the distance learning age”. Birmingham 16.10.2000
Coherence Clusters, bundles, lexical sets, chain interactions, priming Visual sets
Coherence 1: clusters • According to Scott “words which are found repeatedly in each other’s company” (cf. collocation) are clusters.
Coherence 2: bundles Biber (1999) refers to lexical bundles. i.e. the identification of groups (bundles) of lexico-grammatical elements that co-occur with more than usual frequency in particular genres eg. newspaper editorials (present tense, deontic modals, etc.)
Coherence 3: lexical sets • At the word level, lexical sets can be identified within a single text, identified in terms of repetition, synonymy, hyponymy, meronymy or any other semantic relation.
Coherence 4: clusters and bundles • “Clusters and bundles (and lexical sets) can provide clues as to the frequently recurring themes of a discourse” (Partington & Morley, forthcoming)
Visual sets • Similarly, in multimodal texts, visual items will re-occur and co-occur in the same way, and in an integrated relationship with the verbal elements, forming ‘clusters’ of semiotic modalities. • See MCA for an example of a relational database designed to promote the analysis of this phenomenon.
Chain interactions See Hasan, Fries. Chain interactions show how topics are developed and represented in repeated or similar lexico-grammatical forms.
Priming • Clusters and bundles can be ‘primed’ (Hoey) to appear only in specific parts of a text or specific circumstances of discourse production. • … I love you too.
Case Study • Bundaberg Beer ads
