STEP: a Scripting Language for Embodied Agents

1. STEP: a Scripting Languagefor Embodied Agents Zhisheng Huang, Anton Eliens,and Cees Visser 연세대학교 컴퓨터과학과로봇 공학 특강 2004 2학기 10410898 유 지 오

2. Agenda • Introduction • Principles • The Scripting Language STEP • Reference Systems • Primitive Actions and Composite Operators • Dynamic Logic • High-Level Interaction Operators • Examples • Walk • Run • Look at ball • Touch • Related Work • Conclusions

3. Introduction • Interaction  multi-modal communicative acts • Verbal • Non-verbal • Gestures, postures, facial expressions • Specifying communicative acts is not easy • Scripting language STEP (Scripting Technology for Embodied Persona) • Focus on specification and modeling of gestures and postures for 3D web agents • Specification of communicative acts can be separated from the programs  Advantage!! • For H-anim-based humanoids in DLP (based on WWW) • Prolog-like syntax

4. Principles • Design principles For External-oriented communicative acts • Principle 1: Convenience • Hide geometric difficulties • The authors who have limited knowledge of C.G. can use it in a natural way • Example • Turn a left arm forward slowly • turn(Agent, left_arm, front, slow) • More specifty… • Turn(Agent, left_arm,rotation(1,0,0,1.57), 3) • In STEP • Uses some natural-language-like terms for references

5. Principles Principle cont’d • Principle 2: Compositional Semantics • Specification of composite actions based on existing components • Sequence action seq, parallel action par, repeat action repeat • Examples • Turn arms forward slowly • par([turn(Agent, left_arm, front, slow), turn(Agent, right_arm, front, slow)]) • In STEP • A set of built-in action operators • Principle 3: Redefinability • Scripting actions can be defined in terms of other action explicitly • Example • Run then kick • script(run_then_kick(Agent), Action) :- Action = seq([run(Agent), kick(Agent)]) • In STEP • Rule-based specification system

6. Principles Principle cont’d 2 • Principle 4: Parameterization • Adapted to be other actions • Can be specified in terms of how they cause changes over time to each individual degree of freedom • Example • When define the action “run” with respect to a degree of freedom “tempo” • In STEP • Prolog-like syntax • Principle 5: Interaction • Should be able to interact with the world, including objects and other agents • In STEP • Powerful meta-language

7. y x z The Scripting Language STEP Reference Systems • Direction reference • Natural-language-like direction reference scheme Combination of the directions for left arm Direction reference for humanoid

8. The Scripting Language STEP Reference Systems cont’d • Body reference • According to the H-anim standard • All body joints are contained in a hierarchical structure Elbow joint in different situations Typical joints for humanoid

9. The Scripting Language STEP Reference Systems cont’d 2 • Time reference • Same time reference system as VRML • Flexible time reference system based on.. • Beat  interval for body movement • Tempo  number of beats per minute

10. The Scripting Language STEP Primitive Actions and Composite Operators • turn(Agent, BodyPart, Direction, Duration) • Direction: natural-language-like direction or rotation value • Duration: speed name or explicit time specification • move(Agent, BodyPart, Direction, Duration) • Direction: natural-language-like direction or increment value • turn_body(Agent, Direction, Duration) • move_body(Agent, Direction, Duration) • Sequence operator: seq([Action1, …, Actionn]) • Parallel operator: par([Action1, …, Actionn]) • Non-deterministic choice operator: choice([Action1, …, Actionn]) • Repeat operator: repeat(Action, T)

11. The Scripting Language STEP STEP and Dynamic Logic • STEP is based on dynamic logic • Allows for arbitrary abstractions using the primitive and composition operators • A state  the properties at a paticular moment • An action  a relation between two states • Two sub-languages in dynamic logic • Sub-language for actions: a • Sub-language for states: Ψ, Φ

12. The Scripting Language STEP High-Level Interaction Operators • Scripting actions can directly interact with • Internal states of embodied agents • External states of worlds • Lower case greek letters Φ, Ψ, Χ denote formulas in the meta language • test(Φ) : if Φ holds then skip, otherwise fail • do(Φ) : make the state Φ true • If_then_else(Φ,action1,action2) • until(action, Φ) : perform action until Φ holds • All temporal relation can be defined in STEP • 13 possible temporal relations between two actions • Before, meets, overlaps, starts, during, finishes, equals and their inverse relations • Example • overlap(A1,A2)=par([A1,seq([duration(A1,T1),do(random(R)), do(N is T1*R),wait(N),A2])])

13. Examples Walk script(walk_pose(Agent), Action):- Action=seq( [par( [ turn(Agent,r_shoulder,back_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_shoulder,front_down2,fast), turn(Agent,l_hip,back_down2,fast)] ), par( [turn(Agent,l_shoulder,back_down2,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_shoulder,front_down2,fast), turn(Agent,r_hip,back_down2,fast)] )] ). The walk action script(walk_forward_step(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,front,fast)] ). script(walk_forward_step07(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0,0.7),fast)] ). script(walk_forward_step0(Agent,StepLength), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0, StepLength),fast)] ). Walking posture + moving action script(walk_forward(Agent,StepLength,N), Action):- Action=repeat(walk_forward_step0(Agent,StepLength),N). Walking forward N steps

14. Examples Run • Based on the action “walk” script(basic_run_pose(Agent), Action):- Action=par( [turn(Agent,r_elbow,front,fast), turn(Agent,l_elbow,front,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ). Start pose (b) script(run_pose(Agent,N), Action):- Action=seq([basic_run_pose(Agent), repeat(walk_pose(Agent),N)] ). Run action (a) script(run(Agent,StepLength,N), Action):- Action=seq([basic_run_pose(Agent), walk_forward(Agent,StepLength,N)] ). Run action with specifying Steps script(basic_run_pose_elbow(Agent,Elbow_Angle), Action):- Action=par( [turn(Agent,r_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ).

15. Examples Look at Ball • Interaction with Virtual Worlds • Calculate angle script(turn_to_direction(Object,ScrVector,DestVector), Action):- Action=seq( do(vector_cross_product(SrcVector,DestVector,vector(X,Y,Z),R)), do(setRotation(Object,X,Y,Z,R))] ). script(look_at_position(Agent,X1,_Y1,Z1), Action):- Action=seq( [do(getPosition(Agent,X,_Y,Z)), do(Xdif is X1-X), do(Zdif is Z1-Z), turn_to_direction(Agent,vector(0.0,0.0,1.0), vector(Xdif,0.0,Zdif))] ). script(look_at_ball(Agent,Ball), Action):- Action=seq( [do(getPosition(Ball,X1,Y1,Z1)), look_at_position(Agent,X1,Y1,Z1)] ).

16. Examples Touch • An inverse kinematics problem script(getABvalue(Agent,position(X0,Y0,Z0),Hand,A,B), Action):- Action=seq( [ getDvalue(Agent,position(X0,Y0,Z0),Hand,D), get_upperarm_length(Agent,L1), get_forearm_length(Agent,L2), do(D1 is L1+L2), if_then_else(sign(D1-D)>sign(0.001-D), seq( [do(cosine_law(L1,L2,D,A)), do(cosine_law(L1,D,L2,B))] ), seq( [do(A is 1.57*(1+sign(D-0.001))), do(B is 0.0)]))]). α=π and β=0 script(getVvalue(Agent,position(X0,Y0,Z0),Hand,V), Action):- Action=seq( [ get_shoulder_center(Agent,Hand,position(X2,Y2,Z2)), do(direction_vector(position(X2,Y2,Z2),position(X0,Y0,Z0),V))] ).

17. Examples Touch cont’d script(touch(Agent,position(X0,Y0,Z0),l), Action):- Action=seq( [ getABvalue(Agent,position(X0,Y0,Z0),l,A,B), do(R1 is 3.14-A), getVvalue(Agent,position(X0,Y0,Z0),l,D), get_arm_vector(Agent,l,V0), do(vector_cross_product(V0,V,vector(X3,Y3,Z3),c)), do(R2 is C-B), par( [turn (Agent,l_shoulder,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_elbow,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_wrist,rotation(X3,Y3,Z3,-0.5),fast)])]). script(touch_absolutePosition(Agent, postion(X1,Y1,Z1),Hand), Action):- Action=seq( [do(getPosition(Agent,X,Y,Z)), do(getRotation(Agent,X2,Y2,Z2,R)), do(X3 is X1-X), do(Y3 is Y1-Y), do(Z3 is Z1-Z), do(R1 is –R), do(position_rotation(position(X3,Y3,Z3), rotation(X2,Y2,Z2,R1), position(X4,Y4,Z4))), touch(Agent,position(X4,Y4,Z4), Hand)]).

18. Related Work • Perlin and Goldberg’s Improv system • A system for scripting interactive actors in virtual worlds • Compare with STEP… • STEP is based on the H-anim specification • STEP is design to separate the scripting language from the agent architecture • Badler et al ‘s PAR • Similar to • Supports the specification of complex actions for agents • Different.. • PAR uses a special syntax and has many built-in notions

19. Related Work con’d • Funge’s CML (Cognitive Modeling Language) • CML is not VRML/X3D and H-anim based • CML is based on first-order predicate logicvs. STEP is based on dynamic logic • Prendinger’s MPML, SCREAM • Higher level concepts such as affect and social context

20. Conclusions • Proposed the scripting language STEP for embodied agents, in particular for their communicative acts, like gestures and postures • Can be extended for other communicative acts like facial expressions or speech