Using Ultrasound for Language Documentation

1. Using Ultrasound for Language Documentation Amanda L. Miller Cornell University and the University of British Columbia amandami@interchange.ubc.ca

2. Advantages of Ultrasound • Ultrasound is safe, portable and non-invasive. • The first safe tool that allows us to image the back of the tongue, including the tongue root. • Allows us to view the majority of the tongue (difficult to image the tongue tip), and thus will allow us to understand tongue shape better.

3. Difficulties with ultrasound • What we see is a “floating tongue” with no reference to hard bony structures of the head, neck or spine, which make it difficult to interpret images. • This means that tongue movement seen can be due to head movement or ultrasound probe movement. • It is difficult to image the tongue tip with ultrasound (though with some speakers it is possible).

4. Two types of lingual ultrasound • Mid-sagittal ultrasound – have the probe aligned lengthwise under the chin, to view a central slice of the tongue length – can be used to determine the part of the tongue used (tip, blade, front, dorsum, root). • Coronal ultrasound – have the probe turned sideways to view a coronal section of the tongue (difficult to get the same slice on different occasions, difficult to know where in the tongue the image has been taken) – can be used to determine if a sound is lateral or not.

5. Imaging the palate • The ultrasound rays reflect at the first interface with air, so a mid-saggital ultrasound image shows a white line which is the upper edge of the tongue, from the tongue tip to the root. • The palate can typically not be seen in the images, except when the tongue touches the palate. • In order to trace the palate, you can record an ultrasound movie of a swallow (have the subject drink water, hold it in his / her mouth, and swallow). • Different points of the palate can be seen in different frames of the movie, but one composite palate trace is made. • One issue is that the soft palate can move during the production of uvular sounds.

6. Ultrasound of a swallow from a Grootfontein !Xung speaker (/XaiTsubeb)

7. The “floating tongue” problem • You can either stabilize the head and probe to avoid any movement other than that due to speech , or track movements of the head and probe and correct the tongue trace for these movements.

8. Types of ultrasound machines • Sonosite Titan • the smallest, most portable machine • Limited to scanning at 30 fps • Fairly good image quality.

9. Types of ultrasound machines • GE Logiq E scans at frame rates up to 125 fps • Stellar image quality – incorporates many image quality buttons found in larger machines into a portable machine. • Wide scan angle allows researchers to view the whole tongue

10. Types of ultrasound machines • Terason T3000 machine allows collection of ultrasound data and audio at frame rates up to 70 Hz • Uses a script to align the audio signal with the ultrasound video

11. Types of ultrasound machines • Interson’s USB probe called “See More” http://www.interson.com/Products/SeeMore153USBProbes/tabid/79/Default.aspx • Ultra portable (just need the probe and plug it into your laptop) • Affordable ($5-6K) • Only 15 fps frame rate (can do sonorants, vowels)

12. Head and probe stabilization: HATS

13. Probe Stabilization • Magic arm holds probe in place under the person’s head. • Microphone stand with telescoping boom arm holds probe in place under the person’s head. • Ultrasound stabilization headset holds probe in place under the person’s head, and locks in imaging sweet spot over the recording session.

14. Head Stabilization • Moldable Head Stabilizer from the Comfort Company • Board on chair with pillow – ask the subject to keep their head on the pillow

15. Head and Probe Movement Correction • Haskins Optically Corrected Ultrasound System (HOCUS) – not portable. • Palatoglossatron method and software – use two blue sticks with pink dots on them. Software tracks the movement of the pink dots, and corrects the tongue image for the movements in each frame.

16. Palatoglossatron method and Ultrasound stabilization headset

17. Mixing with the audio signal • In most systems, there is no way to record the audio signal into the ultrasound machine. • The Terason T3000 can do this, because it is a laptop.

18. 30 fps ultrasound • Use an Audio – Video Mixer, such as the Canopus TwinPact100, which has frame locking, to mix the audio speech signal and the ultrasound video signal • The ultrasound video signal comes out of the machine either through the S-video port (Sonosite Titan), or the External monitor port (GE Logiq E) • Can be used for monophthongs, sonorants and fricatives, which have fairly stable articulations. • There is a delay in the mixing between the video and audio signals (2-4 frames).

19. CHAUSA • Corrected • High frame rate • Anchored • Ultrasound • Software Alignment

20. CHAUSA Hardware Architecture

21. CHAUSA Software

22. Ultrasound Stabilization Headset, without head movement correction

23. Post-head movement correction

24. Alignment • Articulatory to Acoustic Alignment achieved manually, by aligning 3 click bursts, and 3 [k] bursts in each file. • Alignment was verified using a custom built Tri-Modal 3 ms pulse generator (designed by Engineer, Kenneth Finch, specifications achieve simultaneity to thousands of a millisecond)

25. Proof of Alignment

26. Grootfontein !Xung Alveolar click

27. Introduction • Maddieson (1993) found tongue dorsum retraction following labial-velars in Ewe using EMA data (but sensor coils could not be placed far enough back to view the tongue root with EMA)

28. Hypothesis • Ingressive airstream that is involved in the articulation of Dagbani labial-velars results in tongue dorsum / tongue root retraction. • Labial-velar fronting in Dagbani is the repair strategy found for the constraint • *C [TD/TR retraction] V [coronal]

29. Experimental Goals • To document labial-velar fronting in Dagbani • To see if there is tongue dorsum and tongue root retraction as part of the production of labial-velars in Dagbani in the [ɨ] context.

30. Dagbani labial-velars ISP

31. Mangetti Dune !Xung clicks • Needed High frame rate ultrasound, as 30 fps ultrasound showed high degree of under-sampling (aliasing) • Ultrasound is perfect because it allows us to view the tongue dorsum / tongue root.

32. Experimental Goals • We investigate the rarefaction gestures in all four clicks in Mangetti Dune !Xung (Miller, Scott, Sands and Shah 2009), as well as the C-V transitions in click- high front vowel sequences, to determine whether tongue root retraction is present in the rarefaction gestures, and whether this is carried over into the following vowel.

33. Dental click – Rarefaction gesture Rarefaction gesture displays tongue body lowering, but not tongue root retraction.

34. Alveolar click – Rarefaction gesture Rarefaction gesture involves simultaneous tongue body lowering & Tongue root retraction between Trace 3 and Trace 5

35. Lateral click – Rarefaction gesture Rarefaction gesture displays both tongue body lowering and tongue root retraction.

36. Palatal click – rarefaction gesture There is tongue body lowering. Although the posterior constriction is far back, there is no retraction of the root proper during the rarefaction gesture.