Tracking

1. Tracking Sherman & Craig, pp. 75-94. Welch, Greg and Eric Foxlin (2002). “Motion Tracking: No Silver Bullet, but a Respectable Arsenal,” IEEE Computer Graphics and Applications, special issue on “Tracking,” November/December 2002, 22(6): 24–38.. (http://www.cs.unc.edu/~tracker/media/pdf/cga02_welch_tracking.pdf)

2. Motivation • Methods to interact with the virtual world • More natural • Higher level of immersion • Task performance • Control navigation • Control interaction • Ex. Training soldiers w/ a gun • How do we track the gun? • How do we determine what the user sees? • This requires: • Signaling (button presses, etc.) • Location. <- this is tracking!

3. Tracking • http://www.sv.vt.edu/future/vt-cave/apps/#detour • Pose • Position • Orientation • What do we want to track? • Head pose • Hand pose • Other body part • Other objects (e.g. spider) • So what does it mean if a tracking system reports your head at 2.5,3.3, 1.9?

4. Common Tracking Methods • GPS • Wii http://www.directionsmag.com/images/articles/GPS_articles/realtime_diff_GPS.jpg

5. Y Z X Basic Idea Trackers provide location and/or position information relative to some coordinate system. (x,y,z) (rx,ry,rz) (0,0,0) Receiver coordinate system (0,0,0) Origin for tracker coordinate system

6. Degrees of freedom • The amount of pose information returned by the tracker • Position (3 degrees) • Orientation (3 degrees) • There are trackers that can do: • only position • only orientation • both position and orientation

7. Question • Given that I want to track your head, I attach a new tracker from NewTracker Corp. it returns 6 degrees of freedom (6 floats). What questions should you have? • What are some evaluation points for a tracking system?

8. Data returned Spatial distortion (accuracy) Resolution Jitter (precision) Drift Lag Update Rate Range Interference and noise Mass, Inertia and Encumbrance Number of Tracked Points Durability Wireless Price Evaluation Criteria Which of these are most important?

9. Performance Measures • Registration (Accuracy) – • Difference between an object’s pose and the reported pose • Location • Orientation • What are determining factors? • Resolution • Granularity that the tracking system can distinguish individual points or orientations • What are determining factors? • Jitter • Change in reported position of a stationary object • What are determining factors? • Drift • Steady increase in error with time • What are determining factors?

10. Performance Measures • t0– time when sensor is at point p • t1– time when sensor reports p • Lag or Latency – t1 -t0 • What makes up latency? • Acquisition • Transmission • Filtering

11. Performance Measures • t0– time when sensor is at point p • t1– time when sensor reports p • Lag or Latency – t1 -t0 • What makes up latency? • Acquisition • Transmission • Filtering • What is a minimum?

12. Update Rate • Number of tracker position/orientation samples per second • High update rate != accuracy • Poor use of update information may result in more inaccuracy • Communication pathways and data packet size are important

13. Range • Working volume • What is the shape? • Accuracy decreases with distance • Range is inversely related to accuracy • Position and orientation range could be different • Sensitivity not uniform across all axis

14. Interference and Noise • Interference - external phenomenon that degrades system’s performance • Each type of tracker has different causes of interference/noise • Occlusion • Metal • Noise • Environmental (e.g. door slamming, air conditioner)

15. Mass, Inertia and Encumbrance • Do you really want to wear this? • Inertia • Tethered

16. Multiple Tracked Points • Number of potentially tracked points • Unique • Simultaneous • Difficulties • Interference between the sensors • Multiplexing • Time Multiplexing – Update rate of S samples per second and N sensors results in S/N samples per sensor per second • Frequency Multiplexing – Each sensor broadcasts on a different frequency. More $$

17. Price • You get what you pay for. ($30-$100k+) • Rich people are a small market.

18. Tracking Technologies • Different Tracking Technologies • Goals: • Understand how they work • Understand tradeoffs • Know when to use which • Future directions

19. Mechanical Linkage • Rigid jointed structure • One end (base) is fixed • The other (distal) is free • Distal is user controlled to an arbitrary position and orientation. • Sensors at the joints detect the angle • Concatenate translates and rotates • Determine the position and orientation of the distal relative to the base.

20. Mechanical Tracking • Pros: • Accurate • Fast • Low lag • Minimal environmental interference • No calibration • Can incorporate force feedback • Cons: • Low range (effectively 5’ – does not scale well) • Cost • 1 tracked point (body/others are hard to track) • Data returned: 6 DOF • Spatial distortion – 0.3381 mm • Resolution – very high • Jitter (precision) – very low • Drift - none • Lag – >5ms • Update Rate - 300 Hz • Range - 8 ft • Number of Tracked Points – 1 • Wireless - no • Interference and noise – metal, earth • Mass, Inertia and Encumbrance – substantial • Durability – low • Price – high

21. Mechanical Tracking Products • Fake Space Labs BOOM Display (discontinued) • Sensible Phantom

22. Electromagnetic Trackers • Emitter • Apply current through coil • Magnetic field formed • 3 orthonormal coils to generate fields • Sensor • Strength attenuated by distance • 3 orthonormal magnetic-field-strength sensors • Determine the absolute position and orientation of a tracker relative to a source. • Polhemus (a.c.) • Ascension (d.c.)

23. Basic Principles of EM Trackers • Pulse the emitter coils in succession • Sensor contains 3 orthogonal coils • For each pulse, sensor measures the strength of the signal its 3 coils (9 total measurements) • Known: • Pulse strength at the source • Attenuation rate of field strength with distance • Calculate position and orientation of the sensor coils

24. EM Trackers • Data returned: 6 DOF • Spatial distortion – 0.6 mm, 0.025° • Resolution – 0.00508 mm, 0.025° / inch from receiver • Jitter (precision) – mm to cm • Drift - none • Lag – reported 4 ms • Update Rate - 120 Hz • Range - 5 ft • Number of Tracked Points – 16 (divides update rate) • Wireless - yes • Interference and noise – metal, earth • Mass, Inertia and Encumbrance - minimal • Durability - high • Price - $4000+

25. EM Trackers • Pros: • Measure position and orientation in 3D space • Does not require direct line of sight • Low encumbrance • Cost • Good performance close to emitter • Lag • Can be built ‘into’ devices • Earth magnetic field good for 3DOF • Cons: • Accuracy affected by • DC: Ferrous metal and electromagnetic fields. • AC: Metal and electromagnetic fields • Operate on only one side of the source (the working hemisphere) • Low range (effectively 5’ – does not scale well) • Calibration • Data returned: 6 DOF • Spatial distortion – 0.6 mm, 0.025° • Resolution – 0.00508 mm, 0.025° / inch from receiver • Jitter (precision) – mm to cm • Drift - none • Lag – reported 4 ms • Update Rate - 120 Hz • Range - 5 ft • Number of Tracked Points – 16 (divides update rate) • Wireless - yes • Interference and noise – metal, earth • Mass, Inertia and Encumbrance - minimal • Durability - high • Price - $4000+

26. EM Tracking • Ascension Flock of Birds • Polhemus Fastrak • Extremely popular • Good for many applications • CAVEs (remove metal) • HMDs • Projection displays • Fishtank

27. Acoustic/Ultrasonic Tracking • Time of Flight Tracking • Emitters • Multiple emitters • In succession, emit sound (record time) • Receiver • Report time of receiving sound • Frequency tuned • Calculate time-of-flight (1000 feet/sec) • Use ultrasonic (high) frequencies • Similar: • EM tracking • Radar/sonar • Phase Coherence tracking • Orientation only • Check phase of received signal

28. Ultrasonic Tracking System Setup How much data does 1 transmitter provide? How much data do 2 transmitters provide? How much data do 3 transmitters provide? Stationary Origin (receivers) Tracker (transmitters) distance1 distance2 distance3

29. Acoustic/Ultrasonic Tracking Characteristics • Pros: • Inexpensive • Wide area • Encumbrance • Cons • Inaccurate • Interference • Requires line-of-sight • Data returned: 3 or 6 DOF • Spatial distortion – low (good accuracy) • Resolution – good • Jitter (precision) – mm to cm • Drift - none • Lag – very slow • Update Rate - 120 Hz • Range – 40’+ (scaling issues) • Number of Tracked Points – numerous (spread-spectrum) • Wireless - yes • Interference and noise – medium, noise, environment • Mass, Inertia and Encumbrance - minimal • Durability - high • Price – cheap to $12000+

30. Ultrasonic Tracking Devices • Logitech • Mattel Power Glove • Intersense • Used as part of hybrid systems

31. Inertial Tracking • Electromechanical devices • Detect the relative motion of sensors • Measuring change: • Acceleration (accelerometers) • Gyroscopic forces (electronic gyroscopes piezo electric) • Inclination (inclinometer) • Frameless tracking • Known start • Each reading updates current position

32. Accelerometers • Mounted on to body parts • Detects acceleration • Acceleration is integrated to find the velocity • Velocity is integrated to find position • Unencumbered and large area tracking possible • Difficult to ‘factor’ out gravity

33. Accelerometer Tracking Errors Suppose a constant error i, so that measured acceleration is ai(t)+ i vi(t) = (ai(t)+ i)dt =  ai(t)dt + it xi(t) =  vi(t)dt = ( ai(t)dt + t)dt xi(t) =  ai(t)dtdt + 1/2 it2 Errors accumulate since each position is measured relative to the last position Estimated 10 degrees per minute. How is this related to drift?

34. Inclinometer Measures inclination Relative to some “level” position Gyroscopes Resist rotation Measure resistance Inertial Tracking

35. Inertial Tracking Systems Characteristics • Pros: • Inexpensive • Wide area • Orientation very accurate • Minimal interference • Encumbrance • Cons • Position poor • Need to recenter • Calibration • Inaccurate over time • Drift • Data returned: 3 or 6 DOF • Spatial distortion – low (good accuracy) • Resolution – good • Jitter (precision) – low • Drift - high • Lag – very low • Update Rate - high • Range – very large • Number of Tracked Points – 1 • Wireless - yes • Interference and noise – gravity • Mass, Inertia and Encumbrance - minimal • Durability - high • Price – cheap

36. Optical Trackers • Use vision based systems to track sensors • Outside-Looking In: • Cameras (typically fixed) in the environment • Track a marked point • PPT tracker from WorldViz (www.worldviz.com) • Older optical trackers • Inside-Looking Out: • Cameras carried by participant • Track makers (typically fixed) in the environment • Intersense Optical Tracker • 3rdTech HiBall Tracker Image from: High-Performance Wide- Area Optical Tracking The HiBall Tracking System, Welch, et. al. 1999.

37. Outside Looking In Optical Tracking • Precision Point Tracking by WorldViz • IR Filtered Cameras are calibrated • Intrinsics • Focal length, Center of projection, aspect ratio • Extrinicis • Position and orientation in world space • Each frame: • Get latest images of point • Generate a ray (in world coordinates) through the point on the image plane • Triangulate to get position

38. Outside Looking In Optical Tracking • What factors play a role in O-L-I tracking? • Camera resolution • Frame rate • Camera calibration • Occlusion • CCD Quality • How does it do for: • Position • stable, very good • Orientation • Unstable, poor • Latency • Cameras are 60Hz

39. Orientation • How to compensate for poor orientation? • Combine with orientation only sensor (ex. Intersense’sInertiaCube) • Also known as: • ‘Hybrid tracker’ • ‘Multi-modal tracker’ • Position: vision • Orientation: inertial

40. Inside-Looking-OutOptical Tracking • Tracking device carries the camera • Tracks markers in the environment • Intersense Tracker • 3rdTech HiBall Tracker Images from: High-Performance Wide- Area Optical Tracking The HiBall Tracking System, Welch, et. al. 1999.

41. HiBall Tracker • Position • Pretty good • Orientation • Very good • Latency • LEPDs can operate at 1500 Hz Six Lateral Effect Photo Dioides (LEPDs) in HiBall. Think 6 cameras.

42. LED Optical Trackers • Sensors • Webcameras • Photodiodes • Track • LEDs • Reflected LED light • Why LEDs? • Easy to track • Grab your webcam and point a remote at it • Super cheap • P5 Glove • Nintendo Wii • WorldViz PPT • Virtual Patients

43. Pros: Inexpensive Wide area Very accurate Cons High quality is very expensive Occlusion Calibration Optical Tracking Review • Data returned: 6 DOF • Spatial distortion – very low (very good accuracy) • Resolution – very good • Jitter (precision) – very good • Drift - none • Lag – moderate • Update Rate – low - high • Range – very large (40’ x 40’ +) • Number of Tracked Points – 4 • Wireless - yes • Interference and noise – occlusion • Mass, Inertia and Encumbrance - moderate • Durability – low - high • Price – cheap to very expensive

44. Hybrid Approaches • Nintendo Wii

45. Angle Measurement Measurement of the bend of various joints in the user’s body Used for: • Reconstruction of the position of various body parts (hand, torso). • Measurement of the motion of the human body (medical) • Gestural Interfaces • Sign language

46. Angle Measurement Technology • Optical Sensors • Emitter and receiver on ends of sensor • As sensor is bent, the amount of light from emitter to receiver is attenuated • Attenuation is determined by bend angle • Examples: Flexible hollow tubes, optical fibers • VPL Data Glove

47. Angle Measurement Technology (cont.) • Strain Sensors • Measure the mechanical strain as the sensor is bent. • May be mechanical or electrical in nature. • P5 Glove $25 (!) • Cyberglove (Virtual Technologies)

48. Joints and Cyberglove Sensors Proximal Inter- phalangeal Joint (PIP) Interphalangeal Joint (IP) Metacarpophalangeal Joint (MCP) Metacarpophalangeal Joint (MCP) Abduction Sensors Thumb Rotation Sensor

49. Angle Measurement Technology (cont.) • Exoskeletal Structures • Sensors mimic joint structure • Potentiometers or optical encoders in joints report bend • Exos Dexterous Hand Master

50. Other Techniques • Pinch Gloves • Have sensor contacts on the ends of each finger