Audio theory and practice for language documentation

1. Audio theory and practicefor language documentation DocLing 2016 David Nathan & Anthony Jukes

2. An epistemology for audio in documentation • an audio recording is made in order to be experienced by a human listener • a recording conveys what a human listener would experience at a particular location in an event setting • documentation goals define recording methodology • a recording should capture spatial information • metadata about the recording and the recording setting are required for full interpretation • ethical recording respects speakers and honours their contribution through your effort and skill

3. Evaluating recordings • accuracy: how well is the signal captured, as true to its sources and without distortion? • intelligibility/information accessibility: can the desired content be identified? • signal vs. noise: is the ratio acceptable? can the focal source be separated from all sources of noise? • listenability/comfort/aesthetics: is it easy on the ears? will it be comfortable to listen to for an extended time?

4. Evaluating recordings • localisation of sources: is enough spatial information captured? • separation of noise: can all sources of noise be separated? • representation of environment: are the acoustic properties of the recording space appropriately represented?

5. Evaluating recordings • content (identity, performance, uniqueness, coverage): were the right people recorded doing the right things? • editability/repurposeability: is the recording suitable for turning to relevant purposes?

6. Recording audio • making it is both art and science • a critical and ethical responsibility • strongest relationship to communities • it’s not necessary to record everything, but it is necessary to record well

7. SIGNAL & NOISE

8. Evaluating recordings • signal • noise • signal to noise ratio • listenability (eg comfort, consistency) • fit for purpose

9. Evaluating recordings • audio professionals use their human ears as evaluator of audio quality and value, while many linguists (mistakenly?) look to formats, spectrographs, wave-forms, analyses etc 44.1 KHz, 24 bit  

10. Signal - what you want • content • contextual and spatial information • fidelity • comfortable to listen to

11. Noise - what you don’t want • from environment: • near: people, animals, activities • far: traffic, generators, planes • machines: refrigerators, fans, computers • not hearable: mobile phones, electrical interference • acoustic: reflections/resonance

12. Noise - what you don’t want • generated by unwanted parts of event • shuffling papers, clothes • table banging • backchannel from interviewer • equipment handling, especially microphones and cables (and recorders with built-in mics)

13. Avoiding handling noise • use stands and cradles etc

14. Noise - what you don’t want • generated by equipment • wrong input levels • circuity noise (cheap or incompatible) • compression loss or distortion • ALC/AGC effects (pumping) • video camera motors

15. External noise sources

16. Dead cat

17. Close-up noise sources • machines

18. Dealing with noise sources • be prepared and aware • seek collaboration • monitor • use or modify room acoustics • location • direction • surfaces • reflection • absorption • isolation

19. Utilising room acoustics • location • away from doors, windows, traffic areas • direction • face away from noise sources • reflection • avoid parallel surfaces • surfaces • avoid hard smooth surfaces • choose or create soft or rough surfaces • isolation • find an ‘’airtight’’ place

20. When is a noise not a noise? • When it is part of the content, for some interpretation of the event Performance of John Cage 4”33’ Available on iTunes (150 yen)

21. PERCEPTION & PSYCHOACOUSTICS

22. Audio perception/psychoacoustics • a human listener has: • location, orientation in a physical setting • two ears - incredibly sensitive • a brain/mind • the mind selects from various sources of sound and other sensory information, using long- and short-term memory • listening is actually a “hallucination”

23. Psychoacoustics and recording • microphones don’t have a mind: they can't distinguish wanted from unwanted sound • microphones don’t have “edges” like camera lenses

24. Psychoacoustics and recording • the recording process loses acoustic information • if you only care about transcription, then you are going to throw away over 99% of the acoustic information anyway! real world record acoustic phenomena represent (some) linguistic components derive data

25. Implications for recording • typical recording methods are unscientific! • … so what should we do?

26. Implications for recording • plan and manage recording • goals • equipment preparation and settings • other preparation • environment and setup • sources • changes, actions, settings

27. Implications for recording • why is it important to record spatial information? • what other information (acoustic or non-acoustic) do we need?

28. “Sound stage” • spatial information is an essential part of audio • we are amazingly attuned to it • we should record in stereo

29. “Sound stage” • ... or ORTF (binaural)

30. MICROPHONES

31. Microphones and audio quality • microphones are the greatest factor in audio recording quality • selection of appropriate microphone(s) for the task • placement and handling

32. Microphone types • principle: dynamic vs condenser • directionality: omni, cardoid, and shotgun • spatiality: mono, stereo, ORTF, binaural

33. Microphone physical principles • dynamic • generate signal from sound pressure • more robust, less accurate • used for musical and live performance • condenser • more fragile, sensitive and accurate • need power source - battery or phantom power • in general, use condenser microphones for language documentation

34. Omni • lavalier or tie-clip microphones are typically omni-directional

35. Microphone directionality - omni omni-directional

36. Cardioid • many “standard” handheld microphones are cardioid units

37. Microphone directionality - cardioid cardioid

38. Shotgun (= directional, hypercardioid) • shotguns are good for • quiet sources • in some noisy environments • video work

39. Microphone directionality - shotgun shotgun/directional/hypercardioid

40. Head-mounted microphones • head-mounted microphones are excellent for very noisy environments or mobile activties, and may be omni-directional or cardioid

41. Stereo microphones • spatial information is an important part of audio

42. Full “sound stage”: ORTF Superlux S502 Full binaural on dummy head

43. ORTF & Binaural ORTF is now the “best practice” for field recordings” (Austrian Academy of Science, Vienna Phonogrammarchiv)

44. 110° 17cm Simulating ORTF with 2 cardioids

45. Microphones - quality • generally, you get what you pay for • each model has its own subjective “colour” • good microphones for language documentation cost from US$180 to US$500

46. Reputable makers - examples • AKG • Audio Technica • Beyerdynamic • Røde • Sennheiser • Shure • Sony

47. Microphone placement

48. Microphone usage principles • where should the microphone be? • in general, about 20cm from the speaker’s mouth • the inverse square law is your friend ...

49. The inverse square law

50. Using the inverse square law • if you have noise sources, increase the signal to noise ratio by: • placing the microphone as close as possible to the signal source • placing the microphone as far as possible from the noise source