Development Of Simulation and VR

1. Development Of Simulation and VR Human Dynamics in a Virtual World

2. Recap Initialize world Calculate Geometry Draw Wire Frame Render Surfaces Enhance Surfaces and lighting Sensor input and output

3. Human Dynamics • Users described as participants • basic interaction involves control of camera (viewpoint) • exploratory navigation / locomotion • Walk through systems • More advanced environment allow interaction • Touch , selection, manipulation • referred to as direct manipulation

4. Components of interaction • VR model • Simulation of body • Interaction with virtual body • Object pair collision • General collision detection

5. VR Model • Goal of Being There • Presence or Telepresnce • Held and Durlach 1992, Draper 1998 • Must model expectations -> realism • Ideal VR model must Immerse participant in visual, audio, touch , smell and taste • Humans can process several audio streams and can focus and segrgate on one - Wenzel 1992

6. VR model - Immersion • Surrounds body • fills visual field • extensive • inclusive (replaces reality) • Vivid • human body • in CAVE actual body can obscure projection of virtual objects • In HMD body must be represented

7. VR model - HCI • Mouse and keyboard has two problems • gulf of execution • gulf of evaluation • Hutchins 1986 • Direct Manipulation paradigm • Tracked HMD is simplest form 0- 1 to 1 mapping, Low cognitive overhead • Using mouse - must map actions to different translations

8. VR Model - Interaction • Immersion and tracking rely on registration • Registration implies that motion of limbs accurate • Better appreciation of 3D environment • Cannot lose interaction - reduces gulf of execution • Gulf of evaluation reduced when whole virtual body used - Slater and Usoh 1994, Mine 1997

9. Simulation of Body • Body model is the description of the interface • eyes are viual interface, ears are audio interface • geometric description drawn from egocentric point of view • description of hand and fingers forms basis of grasping simulation for picking up objects (Boulic 1996)

10. Simulation of Body- Building the body • The more points represneting the body the more realistic the movement • Up to 90 points for motion-capture in animation • Standard for human skeleton (H-Anim 1999) • More typically head, Torso, Both hands • Inferred movement from limited points • Inverse kinematics problem - infinite possibilities of movement in virtual environment, consistent restraint • Elbow position in 4- Tracker system (Badler, 1993)

11. H-Anim Humanoid L Hip L Knee L Ankle L Midtarsal Sacroiliac R Hip R Knee R Ankle R Midtarsal L Shoulder L Elbow L Wrist vl5 R Shoulder R Elbow R Wrist Skullbase

12. Simulation Of body - tracking the participant • Choice of system depends on 5 factors • accuracy, resolution, range, lag, update rate • Many different tracking technologies • Meyer 1992 • frequency and time • ultrasonic time-of-flight measurement • Pulsed Infra-red • GPS • Optical Gyroscopes • Phase difference

13. Simulation Of body - tracking the participant • Spatial Scan • Outside-in • Inside-out • Inertial sensing • mechanical gyroscope • Accelerometer • Mechanical Linkages • Direct - Field Sensing

14. Interaction with virtual Body • Limitations mean reliance on metaphors for • object manipulation (grasping and moving) • locomotion (movement) • Limitations in haptics mean that restraint on the virtual environment exists

15. Virtual Bodies • VBs are represenetations of human involvement in 3D applications and VR • Hierarchical connected geomtry specifications • Principles similar to robotics

16. What is a robot? • Joseph Engelberger, a pioneer in industrial robotics, once remarked "I can't define a robot, but I know one when I see one." • Many different machines called robots • Everybody has a different idea of what constitutes a robot • Name from robota – forced labour

17. What relevance to us? • VR models use robotic principles • Avatars behave like robots • Simulations of robots used to test real robots • May be used to control remote robotics

18. Robot Arm • Fitted with end effector • Usually interchangeable • Artificial Hand , paint gun, welding rod • Pressure sensor needed to prevent crushing • Programmed by incremental steps which are then replicated ad infinitum

19. Frameworks, Chains (or Skeletons) • A lot of mechanical objects in the real world consist of solid sections connected by joints • Obviously robot arm but also • Creatures such as humans and animals. • Car Suspension • Ropes, string and Chains

20. Frameworks, Chains (or Skeletons) • Sections and joints of robot arm are known as a 'chain‘ • In creatures could be referred to as a skeleton • Moveable sections correspond to bones • Attachments between bones are joints.

21. Frameworks, Chains (or Skeletons) • Motions of chains can be specified in terms of translations and rotations. • Forward Kinematics - From the amounts of rotation and bending of each joint in an arm, for example, the position of the hand can be calculated. • Inverse Kinematics - If the hand is moved, the rotation and bending of the arm is calculated, in accordance with the length and joint properties of each section of the arm.

22. Joint Translation-Rotation • We can use a transform (T) to transform each point relative to the body to a position in world coordinates. • If we want to model both linear and angular (rotational) motion then we need to use a 4x4 matrix to represent the transform

23. ? End Effector Base What is Inverse Kinematics? • Forward Kinematics

24. End Effector Base What is Inverse Kinematics? • Inverse Kinematics

25. ? End Effector Base What does looks like?

26. Infinite number of solutions ! Solution to • Our example Number of equation : 2 Unknown variables : 3

27. Our example • System DOF = 3 • End Effector DOF = 2 Redundancy • System DOF > End Effector DOF

28. Redundancy • A redundant system has infinite number of solutions • Human skeleton has 70 DOF • Ultra-super redundant • How to solve highly redundant system?

29. Iterative solution • Start at end effector • Move each joint so that end gets closer to target • The angle of rotation for each joint is found by taking the dot product of the vectors from the joint to the current point and from the joint to the desired end point. Then taking the arcsin of this dot product. • To find the sign of this angle (ie which direction to turn), take the cross product of these vectors and checking the sign of the Z element of the vector.

30. Goal Potential Function • “Distance” from the end effector to the goal • Function of joint angles : G(q)

31. Goal distance End Effector Base Our Example

32. Limitations • Will G(q) be always zero? • No : Unreachable Workspace • Will the solution be always found? • No : Local Minima/Singular Configuration • Will the solution be always unique? • No : Redundancy

33. Applications to VR/3d Apps • Control of humanoid components • Can be considered complex chains of interconnected geometrical objects • Activities • Grasp • Walk • Collide • Interact with other objects

34. Object Manipulation World World Body B Object O Body B Object O Hand H Hand H Object P Releasing Object P Grasping

35. Object Manipulation • Hand posture may not be tracked - makes grasping difficult • Must establish a point at which union is deemed to have taken place • Moved by repositioning in the scene graph • Robinett and Holloway 1992

36. Locomotion • Tracker has a limited range • Must use locomotion metaphor to move greater distances • Locomotion is on an even plane , virtual terrain may not be • Collision detection can be employed to raise or lower the participant accordingly

37. Object Manipulation • Hand posture may not be tracked - makes grasping difficult • Must establish a point at which union is deemed to have taken place • Moved by repositioning in the scene graph • Robinett and Holloway 1992

38. Locomotion • Tracker has a limited range • Must use locomotion metaphor to move greater distances • Locomotion is on an even plane , virtual terrain may not be • Collision detection can be employed to raise or lower the participant accordingly

39. Directions of locomotion Fly in direction of aim Fly in direction of pointing Fly in direction of gaze Fly in direction of torso

40. Books and Articles: • The Handbook of Virtual Environments (2002), Kay Stanney (ed), Lawrence Erlbaum. • Isdale, J., 1998, What is VR?http://www.isdale.com/jerry/VR/WhatIsVR.html • Kalawsky, R., 1993, The Science of Virtual Reality and Virtual Environments, Addison Wesley. • Rheingold, H., 1991, Virtual Reality, Secker and Warburg, London. • Wilson, J.R., D’Cruz, M., Cobb, S. and Eastgate, R., 1996, Virtual Reality for Industrial Applications, Nottingham University Press.

41. Resources • www.vrweb.com (VR Solutions Company) • www.barco.com/projection_systems/virtual_and_augmented_reality/ • www.sgi.com (VR Solutions Company) • www.ptc.com (free modelling program) • www.sense8.com (trial VR program) • www.crystalspace.com (free Games Engine)