1 / 1

Research themes Coordinated Multi Modal Generation for Virtual Humans

The Virtual Conductor. Research themes Coordinated Multi Modal Generation for Virtual Humans Coordination of multi modal generation with perception input. Real-time coordinated interaction. Dynamic animation, synchronized with extrapolations of a stream of incoming events/observations.

murphy-stephenson
Download Presentation

Research themes Coordinated Multi Modal Generation for Virtual Humans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Virtual Conductor Research themes Coordinated Multi Modal Generation for Virtual Humans Coordination of multi modal generation with perception input. Real-time coordinated interaction. Dynamic animation, synchronized with extrapolations of a stream of incoming events/observations. Conversational Agents: Theoretical Work The Virtual Conductor will serve as a context for theoretical investigation of dialog as a continuous perception/production feedback loop in which both interaction parties are at all times 'sending' as well as 'receiving'. Conducting Applications The Virtual Conductor (VC) may be used for several applications. The most relevant at this point are the use of the VC for rehearsals and for tutoring of conducting students. Music Perception Technology The VC presents a novel context to work on applications of music analysis technology. Activities Analyzing Conducting by Humans To implement the right behavior in a VC we need to know how different intentions are expressed by human conductors, and when. Music Perception Technology To conduct effectively, the VC needs to analyze the music in real time using algorithms implemented and/or adapted from literature. Multi Modal Generation Animation technology, adaptive (re)planning, synchronization, etc… Evaluation with Human Musicians Conduct tests with human musicians to evaluate the interaction models and the perception technology. • Current State • Animated Virtual Conductor • Conducts a group of human musicians • Using 1-, 2-, 3- and 4-beat patterns. • Conducts volume and tempo (changes). • Tempo interaction: prepares, and leads the musicians through, tempo changes or corrections. • Can rehearse problematic passages with the ensemble • The conductor has been evaluated several times with groups of human musicians. The musicians could follow the tempo and dynamic changes of the conductor reasonably well. The conductor could interact successfully with the musicians, correcting their tempo if they played too fast or to slow. Human Media Interaction P. Bos A. Nijholt D. Reidsma Zs. Ruttkay d.reidsma@ewi.utwente.nl • Interacting with a Virtual Conductor, Bos, P., Reidsma, D., Ruttkay, Z.M., and Nijholt, A. (2006) in: Proc. of 5th International Conference on Entertainment Computing, Cambridge, UK. pp 25-30, LNCS 4161 Springer Verlag, DOI 1007/11872320_3 • http://hmi.ewi.utwente.nl/showcases/Human Music Interaction/

More Related