Audio Theory and Practice for Language Documentation

1. Audio Theory and Practicefor Language Documentation Australian Linguistic Society 4 December 2011 ANU, Canberra David Nathan Endangered Languages Archive Hans Rausing Endangered Languages Project SOAS, University of London

2. Topics – audio and recording • Questions • Epistemology • Signal and noise • Perception and psychacoustics • Microphones • Workflow • Connections • Implications for video • Implications for metadata

3. Questions • have you recorded audio? • what else have you done with your audio?

4. Question • are there any special audio-recording considerations for language documentation?

5. Questions • you buy a recorder for $x. A matching microphone should cost: (a) 3x (b) 0.75x (c) 0.3x (d) 0.1x (e) relative cost is irrelevant

6. Question • Digitally recorded audio is better quality than analogue recorded audio because: (a) digital microphones are more accurate (b) digital formats are more accurate (c) digital equipment is newer (d) digital formats capture more information (e) no, digital audio is not better than analogue audio

7. Big questions • what are we actually recording? • what is it for? • what is the role of audio in language documentation?

8. Epistemology for audio in documentation • an audio recording is made in order to be experienced by a human listener • a recording conveys what a human listener would experience at a particular location in an event setting • documentation goals define recording methodology • a recording should capture spatial information • metadata about the recording and the recording setting are required for full interpretation • ethical recording respects speakers and honours their contribution through your effort and skill

9. Evaluating recordings • accuracy: how well is the signal captured, as true to its sources and without distortion? • intelligibility/information accessibility: can the desired content be identified? • signal vs. noise: is the ratio acceptable? can the focal source be separated from all sources of noise? • listenability/comfort/aesthetics: is it easy on the ears? will it be debilitating to listen to for an extended time?

10. Evaluating recordings • localisation of sources: is enough spatial information captured? • separation of noise: can all sources of noise be separated? • representation of environment: are the acoustic properties of the recording space appropriately represented?

11. Evaluating recordings • content (identity, performance, uniqueness, coverage): were the right people recorded doing the right things? • editability/repurposeability: is the recording suitable for turning to relevant purposes?

12. What is audio? • Audio is not data • real world • record acoustic phenomena • represent (some) linguistic components • derive data • Audio is a resource • making it is both art and science • a critical and ethical responsibility • strongest relationship to communities • it’s not necessary to record everything, but it is neceessary to record well

13. SIGNAL & NOISE

14. Evaluating recordings • signal • noise • signal to noise ratio • listenability (eg comfort, consistency) • fit for purpose

15. Evaluating recordings • audio professionals use their human ears as evaluator of audio quality and value, while many linguists (mistakenly?) look to formats, spectrographs, wave-forms, analyses etc   44.1 KHz, 24 bit

16. Signal - what you want • content • contextual and spatial informationSoundscape and ethnography • fidelity • comfortable to listen to

17. Noise - what you don’t want • from environment: • near: people, animals, activities • far: traffic, generators, planes • machines: refrigerators, fans, computers • not hearable: mobile phones, electrical interference • acoustic: reflections/resonance

18. Noise - what you don’t want • generated by (unwanted parts of) event • shuffling papers, clothes • table banging • backchannel from interviewer • equipment handling, especially microphones and cables (and recorders with built-in mics)

19. Avoiding handling noise • use stands and cradles etc

20. Noise - what you don’t want • generated by equipment • wrong input levels • circuity noise (cheap or incompatible) • compression loss or distortion • ALC/AGC effects (pumping) • video camera motors

21. Evaluating environment/situation external environment • access • electricity • external noise sources

22. External noise sources • see also General principles

23. Dead cat

24. Close-up noise sources • machines

25. Dealing with noise sources • be prepared and aware • seek collaboration • monitor • use or modify room acoustics • location • direction • surfaces • reflection • absorption • isolation

26. Utilising room acoustics • location • away from doors, windows, traffic areas • direction • face away from noise sources • reflection • avoid parallel surfaces • surfaces • avoid hard smooth surfaces • choose or create soft or rough surfaces • isolation • find an ‘’airtight’’ place

27. When is a noise not a noise? • When it is part of the content, for some interpretation of the event John Cage performance

28. PERCEPTION & PSYCHOACOUSTICS

29. Audio perception/psychoacoustics • a human listener has: • location, orientation in a physical setting • two ears - incredibly sensitive • a brain/mind • the mind selects from various sources of sound and other sensory information, using long- and short-term memory • listening is actually a “hallucination”

30. Psychoacoustics and recording • microphones don’t have a mind: they can't distinguish wanted from unwanted sound • microphones don’t have “edges” like camera lenses

31. Psychoacoustics and recording • the recording process removes acoustic information • if you only care about transcription, then you are going to throw away over 99% of the acoustic information! • real world • record acoustic phenomena • represent (some) linguistic components • derive data

32. Implications for recording • typical recording methods are unscientific! • … so what should we do?

33. Implications for recording • plan and manage recording • goals • equipment • preparation • environment and setup • sources • changes and actions • settings

34. Implications for recording • why is it important to record spatial information? • what other information (acoustic or non-acoustic) do we need?

35. “Sound stage” • spatial information is an essential part of audio • we are amazingly attuned to it • we should record in stereo

36. “Sound stage” • ... or in ORTF (binaural)

37. MICROPHONES

38. Microphones and audio quality • microphones are the greatest determinant of audio recording quality • selection of appropriate microphone(s) for the task • placement and handling of the microphone(s)

39. Microphones in the digital era • microphones in the digital era • recorder quality has increased but prices decreased • microphones have become comparatively more expensive • why? microphones are analogue devices!

40. Microphone types • principle: dynamic vs condenser • directionality: omni, cardoid, and shotgun • spatiality: mono, stereo, binaural

41. Microphone physical principles • dynamic • generate signal from sound pressure • more robust, less accurate • used for musical and live performance • condenser • more fragile, sensitive and accurate • need power source - battery or phantom power • in general, use condenser microphones for language documentation

42. Omni • lavalier or tie-clip microphones are typically omni-directional

43. Microphone directionality - omni omni-directional

44. Cardioid • many “standard” handheld microphones are cardioid units

45. Microphone directionality - cardioid cardioid

46. Shotgun • shotguns are good for quiet sources, in some noisy environments, and for video work

47. Microphone directionality - shotgun shotgun/directional/hypercardioid

48. Stereo microphones • spatial information is an essential part of audio

49. Full “sound stage”: ORTF

50. 110° 17cm Simulating ORTF with 2 cardioids