30 likes | 186 Views
=0.94. W. M. Irritation. AM. 8. =0.86. Response (V). A. AI. 6. AIM. IM. Predicted rating. 4. I. 100. 200. 300. 400. Pleasantness. Time (samples). True rating. Synchronized audio-visual recording. Novel speech signal. AUDIO PROCESSING. VIDEO PROCESSING.
E N D
=0.94 W M Irritation AM 8 =0.86 Response (V) A AI 6 AIM IM Predicted rating 4 I 100 200 300 400 Pleasantness Time (samples) True rating
Synchronized audio-visual recording Novel speech signal AUDIO PROCESSING VIDEO PROCESSING Extract speech waveform Extract Video frames Acoustic analysis (PCBF, Energy, F0) Pre-processing and windowing Track MPEG-4 facial markers Extract acoustic features Table lookup: Nearest-neighbors in acoustic feature space PCBF TRAINING PHASE Energy 3D reconstruction from stereo F0 3D Animation/Synthesis Articulatory trajectories Width Add context Height RECALL PHASE time Audio-visual lookup table