1 / 18

Toward a Unified Scripting Language : Lessons Learned from Developing CML and AML

Toward a Unified Scripting Language : Lessons Learned from Developing CML and AML. Soft computing Laboratory Yonsei University October 1, 200 4. 1. Introduction. Two approaches to specifying scripting languages for character animation Character Markup Language(CML)

cleary
Download Presentation

Toward a Unified Scripting Language : Lessons Learned from Developing CML and AML

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toward a Unified Scripting Language : Lessons Learned from Developing CML and AML Soft computing Laboratory Yonsei University October1, 2004

  2. 1. Introduction • Two approaches to specifying scripting languages for character animation • Character Markup Language(CML) • Top-down approach by defining high-level attributes for character personality, emotion and behavior that are integrated to form the specification of synchronized animation script • Avatar Markup Language(AML) • Bottom-up approach in which the language provides a generic mechanism for the selection and synchronized merging of animations • Need for powerful yet generic scripting languages to bridge the gap between behavior generation and animation tools

  3. 2. Scripting with the Character Markup Language • Visual Behavior Definition • A third factor that governs character behavior • The actions an agent needs to perform in a session to achieve given tasks • Personality and current mental state • The role the agent is given • The behaviors are defined as XML tags CML Processor High-level behavior tags Animation Script (map appropriate action point parameters)

  4. 2. Scripting with the Character Markup Language • Classification of Motion • Derived from Blumberg and Russell’s architecture (three-layer structure : geometry, motor, behavior system) • The initial set of CML base motions is classified by the goal of the motion as follows: • Movement defines motion that require the rotation or movement of a character from one position to another. Positions are defined by exact coordinates, an object position, or a character position. (move-to, turn-to) • Pointing defines a pointing gesture toward a coordinate, object, or character. (point-to) • Grasping defines motions that require the character to hold, throw, or come in contact with an object or another character (grasp, throw, touch) • Gaze defines the movements related to the head and eyes. (Gaze are gaze, track, blink, look-to, and look-at. ) • Gesture includes motions that represent known gestures like hand movement to convey an acknowledgment, a wave, etc. (gesture-at)

  5. 2. Scripting with the Character Markup Language • CML Specification • Animated character behavior is expressed through the interpretation of XML Schema structures • The language contains • Low-level tags (specific character gesture representations defining movements, intensities, and explicit expression) • High-level tags (commonly used combinations of these low-level tags • Synchronization between the audio and visual modalities is achieved through the use of SMIL ( <seq>, <par>)

  6. 2. Scripting with the Character Markup Language • CML Representation Language • Head Gesture Taxonomy • Symbolic gestures, Iconic gestures, Deictic gesture • Hand Gestures Taxonomy • Posture, Motion, Orientation, ‘Gestlets’, Fingers • Body Gestures Taxonomy • Natural, Relax, Tense, Iconic, Incline • Emotions (based on the OCC theory of emotion) • Class, Valence, Subject, Target, Intensity, Time-stamp, Origin

  7. 2. Scripting with the Character Markup Language • CML Scripting Language • Face Animation Scripting • Head Movement • tilt : movement in a slant with often subtle or superficial neck movement • turn : require more profound movement of the neck <turn-to> <order {0 to n/before/after} /> <priority 0 to n/> <begin {ss:mmm:/before/after/object} /> <end {ss:mmm:/before/after/object} /> <speed {0.n to n.n (unit) /defualt/slow/fast} /> <target {x,y,z/object/character} /> <direction (rightside, leftside, frontside, backside) /> <degree (n%) /> <repeat {0 to n/dur} /> <interrupt {yes/no} /> <transAnimat {head groups} /> <transPos {x,y,z/object/character} /> <transSpeed default/slow/ intermediate/fast} /> </turn-to>

  8. 2. Scripting with the Character Markup Language • Head Gesture • Deictic • Symbolic • Face Movement and Gesture • Define the elements and behaviors for specific parts of the face including the Brow, Gaze, and Mouse <disagree> <tilt-to-right> … </tilt-to-right> <tilt-to-left> … </tilt-to-left> </disagree>

  9. 2. Scripting with the Character Markup Language • Body Animation Scripting • A set of defined body elements which are low level tags based on MPEG-4 BAPs • A set of high-level tags representing body parts which are grouped from a set of respective low-level tags • Base elements are.. • Movement (moving, bending, turning) • Gesture defines body postures that include motions representing common Iconic, Symbolic, or Deictic body gestures • Posture (expression) defines a set of high-level tags representing general body gesture • Natural ,Relax ,Tense, Incline

  10. 2. Scripting with the Character Markup Language BDB • Generating Script PDTD FDB CML Generation Animation Generation EDTD State and Context CML Processor CML Decoder FAP Audio/TTS Face Action Composition Input FAP BAP Audio/TTS Emotion Signal Utterance Composition CML Script Face &Body Behavior/Action Synchronisation Speech Text

  11. 3. Avatar Markup Language • Objective : to design and develop a full end-to-end MPEG-4 multimedia framework to support, amongst other features, 3D avatar-based multi-user chat rooms and autonomous synthetic characters • Three component • A database of basic facial and body animation units(can extended by third party) • A rendering system capable of merging multiple face and body animation units and text to speech input in realtime • A high-level scripting language designed to allow animators to specify which animations to use together with timing, priority, and decay information

  12. 3. Avatar Markup Language • AML Specification • Flexiblity : as many Expression Tracks as required, each containing as many Expressions as required <AFML> <Settings> …… </Settings> … <ExpressionsTrack name=“Track name”> <Expression> <StartTime>mm:ss:mmm</StartTime> <ExName>”name”</ExName> <Envelope> …… </Envelope> </Expression> </ExpressionTrack> … </AFML> <AFML syntax>

  13. 3. Avatar Markup Language <BodyAnimationTrack name=“char, name of track”> … <UserAnimation type=“bap|trk|wrl” filename=“char”> <StartTime> mm:ss:mmm|autosync|autoafter</StartTime> <Speed>normal|slow|fast</Speed> <Intensity>float, 0 to fn </Intensity> <Priority>integer, 0 to n</Priority> </UserAnimation> <Emotion>…</Emotion> <StandardGesture>… </StandardGesture> </BodyAnimationTrack> <ABML’s Body AnimationTrack Syntax> <BodyAnimationTrack name=“char, name of track”> <FacingAction bodypart=“body|headonly”> … <XCoor>float, target’s X coordinate in meters</XCoor> <YCoor>float, target’s Y coordinate in meters</YCoor> <ZCoor>float, target’s Z coordinate in meters</ZCoor> </FacingAction> <PointingAction handconfig=“onefinger|open”>… </ PointingAction> <WalkingAction mode=“default|run”>…</WalkingAction> <ResettingAction>…</ResettingAction> </BodyAnimationTrack> <ABML’s Body AnimationTrack Behavior>

  14. 4. CML and AML Applied • CML • Sample CML – Happy Move and Point Script <cml> <character name=“James” personality=“extravert” role=“psa” gender=“m” base-animation-file=“butler.liv”> <happy intensity=“0.3” decay=“0.5” target=“goal” priority=“1”> <move-to order=“0” priority=“0” speed=“default” object=“product1” /> <sync type=“par” order=“1” priority=“0”> <point-to object=“product1” /> <utterance> </sync> </happy> </character> </cml>

  15. 4. CML and AML Applied • AML • Sample AML – Walk and Point <AML face_id=“x” body_id=“y” root_path=“c:\” name=“point to phone”> <FA start_time=“00:00:000”> <TTS output_fap=speech.fap output_wav=speech.wav> <Text>”Let me show you another phone over here”</Text> </TTS> <AFML> <Settings> <Fps>25</Fps> <Duration>00:06:000</Duration> <FADBPath>.\Expressions\</FADBPath> <SpeechPath>.\Speech\</SpeechPath> </Settings> <ExpressionsTrack name=“smile”> <Expresion> <StartTime> 00:00:800 </StartTime> <ExName> smile.ex </ExName> <Envelope> <Point>… </Point> <Point>…</Point> <Point>…</Point> </Envelope> </Expression> </ExpressionTrack> <SpeechTrack> …</SpeechTrack> </AFML> </FA> <BA> <ABML>….</ABML> </BA> </AML>

  16. 5. Discussion and Lessons Learned • A Comparison

  17. 5. Discussion and Lessons Learned • Towards a Unified Language • Objectives of a Unified Language • Define a framework to decouple embodied agent animation tools and the underlying affect and planning engines • Establish a formal specification for unified /consistent interpretation • Provide for modular development • Create a markup language based on XML (providing semantic and scripting annotations) • Language Requirements • High Level • Usability • Extensibility • Parameterized • Synchronization Support • Consistency • Domain

  18. 6. Conclusion • There is a trade-off to be made between higher levels of control (high granularity) and the complexity of the resulting language • Striving toward a unified scripting and representation language may be the catalyst for much needed agreement

More Related