Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010

1. Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010

2. Using Vocalization Features to Identify Ethanol Intoxication in Rhesus Macaques by Helen Zou July 23, 2010

3. Introduction Background Rhesus Macaques Speech processing Literature review Previous findings in humans Macaque vocalizations Experiment procedure Data analysis Segmentation and clustering Extracting features Results Acknowledgments Overview

4. Duke University – Class of 2013 Biomedical Engineering major and Neuroscience minor Emailed Dr. Grant because of her work with primates and neuroscience Vocalization project Worked at both ONPRC and OGI Not under any specific program, except… Had to give a presentation anyway Introduction

5. Alcohol drug discrimination and self-administration Predictors of heavy drinking (dominance-related?) BEC (Blood Ethanol Concentration) Need simpler way to measure intoxication in social settings Why not look at speech? Background – Rhesus Macaques

6. Voiced, unvoiced, and noise For monkeys, we focused on voiced (coos and screams) Potential features Frequency and pitch Shimmer (amplitude) and jitter (pitch) Spectral entropy Root mean square (energy) Background – Speech Processing

7. Sample Wave Form

8. Sample Voiced Region

9. Sample Noise

10. Sample Background

11. Introduction Background Rhesus Macaques Speech processing Literature review Previous findings in humans Exxon Valdez case Macaque vocalizations Overview

12. Approach recognition of intoxication as speaker identification task Measure laryngeal and articulatory features Laryngeal - fundamental frequency and signal-to-noise (SNR) Articulatory – formants (F1/F2 ratio) Major findings Increased FO variation Decreased SNR Did not change F1/F2 Limitation: small sample size Much more accurate than human recognition Prior Studies – KlingholzRecognition of low-level alcohol intoxication from speech signal (1988)

13. Measured several different features Nonfluency increase is best measure F0 increases and utterance duration increases (moderate measure) F0 variability slightly increases (poor measure) Vocal intensity had no change 20% of subjects exhibited no consistent changes Unfortunately, disagrees with the previous findings Prior Studies – HollienEffects of ethanol intoxication on speech suprasegmentals (2001)

14. Oil tanker crashed in Alaska in 1989 Captain of ship denied intoxication Analysis of speech found: Misspoken words Slurred pronunciations Slower speaking rate Lower pitch Increased f0 variability Characteristics were consistent with intoxication Exxon Valdez Court CaseAcoustic Analysis of Voice Recordings from the Exxon Valdez by J. Tanford et al (1992)

15. Focused on testing the effect of different social situations Social separation: EtOH reduced isolation peeps Aggression: EtOH increased aggression peeps Social context determines effect of drugs (potential confounding variable?) Previous Study – WeertsPrimate vocalizations during social separation and aggression: effects of alcohol and benzodiazepines (1996)

16. Experiments done on the effect of intoxication on human speech have inconsistent findings Very few studies actually done on macaque vocalizations Many uncontrolled variables (long-term voice effort, social context, etc.) Definitely some effect of ethanol intoxication on speech features Summary of Previous Work

17. Will the vocalizations of monkeys change when intoxicated versus when sober? The Question

18. Put recorders on the monkeys Gavage with water or alcohol (alternating) Measure BECs in one hour Take off recorders Analyze data for various features Identify differences in vocalization Draw conclusions from data and voila! But in reality… Methods

19. Exceeding recorder threshold Not enough vocalizations Problems Solutions • Attenuate with rubber and foam • Switch to more vocal monkey

20. Clementine Example Waveform

21. Recordings had vocalizations, noise, silence, other monkeys, etc. How would we isolate the monkey of interest? Data Analysis?

22. Sample Spectrum Vocalizations Noise

23. Clementine Example Spectrum

24. Cut the wave file into smaller segments Isolate vocalization parts of speech Extract features for vocalization regions Compare features for intoxicated versus sober speech Data Analysis

25. Originally created for separating speakers in news broadcasts Find likely change points Segment data with overlapping frames Cluster similar segments (by speaker) Segmentation/ClusteringRobust Speaker Change Detection by J. Ajmera et al. (2003)

26. Segmentation and Clustering

27. Cut the wave file into smaller segments Isolate vocalization parts of speech Extract features for vocalization regions Compare features for intoxicated versus sober speech Data Analysis

28. Spectrum – Human vs. Monkey

29. Bandwidth of formants in monkey vocalizations is larger than for humans Humans have more formants (5+), monkeys have much fewer (2-4) Distance between the formants for monkeys is much larger than between human formants Shape of formants is curved for screams and straight for coos Results – Human vs. Monkey

30. Spectrum – Human vs. Monkey Human Noise Coo Scream

31. Results - F0 graphs

32. Graphed all of the features F0 as x-variable produced most significant results F0 tends to be higher during intoxication Results – Alcohol vs. Water

33. Results – Rms vs. f0

34. Root mean square (energy) vs. fundamental frequency Control vocalizations have larger variation in energy Intoxication has higher f0 Results

35. Results – Rms vs. Spec entropy

36. Spectral entropy vs. f0 Control vocalizations have larger variation in spectral entropy Intoxication has higher f0 Results

37. Alcohol increases fundamental frequency (agrees with Hollien study) Alcohol decreases variation in energy and spectral entropy Consistent with alcohol impairing muscle control of vocal cords Results

38. Very small sample size Limited number of vocalizations Lots of silence and noise in recordings BEC was low (between .017 and .044) Monkeys were separated – may have different results in social setting Only paired comparisons Limitations

39. Further study correlations between different vocalization features and intoxication Use recordings to correlate with other factors (such as stress, dominance, etc.) Find ways to increase vocalizations Pair vocal recordings with visual tracking Measure ethanol intake using vocalizations in social settings Expand studies to other species In the Future

40. Added to the studies done on macaque vocalizations Used computer algorithms to separate and analyze data Found that formants are a good way to separate human and monkey vocalizations Alcohol increases f0 and decreases variability of energy and spectral entropy Eventually use vocalizations to measure intoxication in macaques in social settings Conclusion

41. Dr. Kathy Grant Dr. Izhak Shafran The Grant Lab (Kevin Nusser, Andrew Rau, Jessica Shaw, and Cara Candell) Meysam Asgari OGI and ONPRC staff and coworkers Acknowledgments

42. Questions?

43. Approach recognition of intoxication as speaker identification task 11 human test subjects and 5 controls Read a text segment in German Measure laryngeal and articulatory features Laryngeal - fundamental frequency and signal-to-noise (SNR) Articulatory – formants (F1/F2 ratio) Intoxication results Increased FO variation Decreased SNR Did not change F1/F2 Correlation between BAL and F0 Long-term voice effort has similar effect Much more accurate than human recognition Prior Studies - Klingholz

44. Speech samples at four levels of intoxication 35 human subjects Results Nonfluency increase is best measure F0 increases and utterance duration increases (moderate measure) F0 variability increases (poor measure) Vocal intensity had no change 20% of subjects exhibited no consistent changes Prior Studies - Hollien

45. 33 squirrel monkeys in two different social situations Social separation: EtOH reduced isolation peeps Aggression: EtOH increased aggression peeps Social context determines effect of drugs Prior Studies - Weerts