Introduction to Tracking

1. Introduction to Tracking Sherman & Craig, pp. 75-94. Welch, Greg and Eric Foxlin (2002). “Motion Tracking: No Silver Bullet, but a Respectable Arsenal,” IEEE Computer Graphics and Applications, special issue on “Tracking,” November/December 2002, 22(6): 24–38.. (http://www.cs.unc.edu/~tracker/media/pdf/cga02_welch_tracking.pdf)

2. Motivation • We want to use the human body as an input device • more natural • this will lead to higher level of immersion • to control navigation • head • hand • to control interaction • head • hand • body • We need two things for this: • Signaling (button presses, etc.) • Location. <- this is tracking!

3. Tracking • Position • Location • Orientation • Pose • Examples • Head position • Hand position (pose) • Other body parts (e.g., self-avatars) • Other objects that also have physical representations (spider).

4. Y Z X Basic Idea Trackers provide location and/or position information relative to some coordinate system. What info would we need? (x,y,z) (rx,ry,rz) (0,0,0) Receiver coordinate system (0,0,0) Origin for tracker coordinate system

5. Degrees of freedom • The amount of pose information returned by the tracker • Position (3 degrees) • Orientation (3 degrees) • There are trackers that can do: • only position • only orientation • both position and orientation

6. Question • Okay, given that I want to track your head, I attach a new tracker from NewTracker Corp. it returns 6 degrees of freedom (6 floats). What questions should you have? • In other words, what are some evaluation points for a tracking system? • 5 minutes to discuss

7. Data returned (3 dof, 6 dof, >6 dof) Spatial distortion (accuracy) (sub mm) Resolution (sub mm) Jitter (precision) (sub mm) Drift Lag (1 ms) Update Rate (2000 Hz) Range (40’x40’ – GPS) Interference and noise Mass, Inertia and Encumbrance Multiple Tracked Points (1-4, 128) Durability (self-contained?) Wireless (yes) Price ($1800 3dof - $40,000+, $180k+ mocap) Evaluation Criteria Which of these are most important?

8. Data returned • Spatial distortion (accuracy) • Resolution • Jitter (precision) • Drift • Lag • Update Rate • Range Performance Measures Reportable location and orientation based on resolution Jitter Drift Registration Actual Object Position

9. Performance Measures • Registration (Accuracy) – Represents the difference between an object’s actual 3D position and the position reported by the tracker • Location • Orientation • Resolution – Fineness with which the tracking system can distinguish individual points or orientations in space. • Jitter – Change in reported position of a stationary object. • Drift – Steady increase in tracker error with time.

10. Performance Measures • Lag (Phase Lag) – Difference between when a sensor first arrives at a point and when the tracking system first reports that the sensor is at that point. Sometimes called latency. • Latency: The rate (or time delay) at which the acquisition portion of the system can acquire new data. • Transmission Lag: Time needed to send bits of information that define position to the computer or graphics engine.

11. Update Rate • Number of tracker position/orientation samples per second that are transmitted to the receiving computer. • Fast update rate is not the same thing as accurate position information. • Poor use of update information may result in more inaccuracy. • Upper bound is determined by the communications rate between tracker and computer and the number of bits it takes to encode position and orientation.

12. Range • Position range or working volume • Sphere (or hemisphere) around the transmitter. • Accuracy decreases with distance • Position range is inversely related to accuracy. • Orientation Range – set of sensor orientations that the tracking system can report with a given resolution.

13. Interference and Noise • Interference is the action of some external phenomenon on the tracking system that causes the system’s performance to degrade in some way. • Noise – random variation in an otherwise constant reading. (Static position resolution) • Inaccuracies due to environmental objects.

14. Mass, Inertia and Encumbrance • Do you really want to wear this? • Things with no weight on your head can have inertia. • Tethered

15. Multiple Tracked Points • Ability to track multiple sensors within the same working volume. • Interference between the sensors • Multiplexing • Time Multiplexing – Update rate of S samples per second and N sensors results in S/N samples per sensor per second • Frequency Multiplexing – Each sensor broadcasts on a different frequency. More $$

16. Price • You get what you pay for. • Rich people are a small market.

17. Position Tracking Orthogonal Electromagnetic Fields Measurement of Mechanical Linkages Ultrasonic Signals Inertial Tracking Optical Tracking Inside Looking Out (Videometric) Outside Looking In Angle Measurement Optical Sensors Strain Sensors Exoskeletal Structures Body Tracking Technology

18. Electromagnetic Trackers • Use the attenuation of oriented electromagnetic signals to determine the absolute position and orientation of a tracker relative to a source. • Polhemus (a.c.) • Ascension (d.c.)

19. Basic Principles of EM Trackers • Source contains 3 orthogonal coils that are pulsed in rotation, one after another. • Each pulse transmits a radio frequency electromagnetic signal that is detected by a sensor. • The sensor also contains 3 orthogonal coils, which measure the strength of the signal from the current source coil (9 total measurements) • By using the known pulse strength at the source and the known attenuation of the strength with distance, these nine values can be used to calculate position and orientation of the sensor coils.

20. Basic EM Principles (cont.) • Source and sensor are connected to a box which contains a microcomputer and electronics associated with the pulses. • Serial communications (serial port) • A source may be associated with 1 to as many as 18 sensors • Problems: Earth’s Magnetism!

21. Characteristics of EM Trackers • Measure position and orientation in 3D space • Do not require direct line of sight between the source and the sensor • Accuracy affected by • DC: Ferrous metal and electromagnetic fields. • AC: Metal and electromagnetic fields • Operate on only one side of the source (the working hemisphere). • Working distance of about 3-25? feet from source. (Depends on source size, power)

22. Output of EM Trackers • Polhemus (AC) • Position: 3 Integers • Orientation: Euler angles,Directional Cosines, Quaternions • Ascension (DC) • Position: 3 Integers • Orientation: Euler angles, 3x3 Rotation Matrices

23. Technology • Electromagnetic Transducers • Ascension Flock of Birds, etc • Polhemus Fastrak, etc • Limited range/resolution • Tethered (cables to box) • Metal in environment • No identification problem • 6DOF Realtime • 30-144 Hz 13-18 sensors

24. Example • 6 bytes for position (3 two-byte integers) • 18 bytes for orientation (9 two-byte integers of a 3x3 orientation matrix). • 3 byte header • 8 data bits and 1 stop bit, no start or parity bits (9 bits/byte) • Total per data packet: 27*9 = 243 bits • 19,200 baud • 13 millisecond transmission time • 79 packets/second • Now all USB

25. Lag between actual and rendered position • Time to acquire and compute position and orientation • Transmission time (0.013 seconds for example for one sensor). • Graphics Frame rate (10-60 frames/sec)

26. Mechanical Linkage • Jointed structure that is rigid except at the joints. • One end (base) is fixed. • The other (free, distal) end may be moved to an arbitrary position and orientation. • Sensors at the joints, detect the angle of the joints. • Concatenation of translates and rotates can be used to determine the position and orientation of the distal end relative to the base.

27. Characteristics of ML • Fast • Accurate: • Depends on the physical size of the ML • Depends on quality of rotation sensors at joints • Encumbered Movement • Expensive • Can incorporate force feedback (PHANToM) • Used on the BOOM display system from Fake Space Labs

28. Sensible Technolgies Phantom

29. Ultrasonic Tracking • Use the time-of-flight of an ultrasonic sound pulse from an emitter to a receiver. Either the emitter or the receiver can be fixed, with the other free to move. • Logitec • Mattel Power Glove • A component of Intersense • Inertial + Ultrasonic systems

30. Basic Principles of UT • Based on measurement of time-of-flight of a sound signal. 1000 feet/Sec • Source component contains transmitters that produce a short burst of sound at a fixed ultrasonic frequency. • The sensor component contains microphones that are tuned to the frequency of the sources.

31. UT Characteristics • Inexpensive (Used in Mattell Powerglove $100). • Inaccurate. • Echoes and other ambient noise • Require a clear line-of-sight between the emitter and the receiver. • Sometimes used for head-tracking for CRT displays.

32. Basic UT Setup Stationary Origin (receivers) Tracker (transmitters) distance1 distance2 distance3

33. UT Position and Orientation Information • 1 transmitter, 3 receivers : 3D position relative to fixed origin • 2 transmitters, 3 receivers : 3D position and orientation up to a roll around a line through the two transmitters • 3 transmitters, 3 receivers : complete position and orientation information

34. Inertial Tracking • Uses electromechanical devices to detect the relative motion of sensors by measuring change in: • Acceleration • Gyroscopic forces • Inclination

35. Accelerometers • Mounted on a body part to detect acceleration of that body part. • Acceleration is integrated to find the velocity which is then integrated to find position. • Unencumbered and large area tracking possible

36. Accelerometer Tracking Errors Suppose the acceleration is measured with a constant error i, so that measured acceleration is ai(t)+ I vi(t) = (ai(t)+ i)dt =  ai(t)dt + it xi(t) =  vi(t)dt = ( ai(t)dt + t)dt xi(t) =  ai(t)dtdt + 1/2 it2 Errors accumulate since each position is measured relative to the last position

37. Inclinometer – measures inclination relative to some “level” position Gyroscopes Inertial Tracking

38. Optical Trackers • Outside-Looking In: • Cameras (typically fixed) in the environment track a marked point. • PPT tracker from WorldViz (www.worldviz.com) • Older optical trackers • Inside-Looking Out: • Cameras carried by participant, tracking makers (typically fixed) in the environment • Intersense Optical Tracker • 3rdTech HiBall Tracker Image from: High-Performance Wide- Area Optical Tracking The HiBall Tracking System, Welch, et. al. 1999.

39. Outside Looking In Optical Tracking • Precision Point Tracking by WorldViz • IR Filtered Cameras are calibrated • Each frame: • Get latest images of point • Generate a ray (in world coordinates) through the point on the image plane • Triangulate to get position

40. Outside Looking In Optical Tracking • What factors play a role in O-L-I tracking? • Camera resolution • Frame rate • Camera calibration • Occlusion • CCD Quality • How does it do for: • Position • stable, very good • Orientation • Unstable, poor • Latency • Cameras are 60Hz

41. Orientation • Since orientation is poor, you can get an orientation only sensor (ex. Intersense’s InertiaCube) • Called a ‘hybrid tracker’ or ‘multi-modal tracker’ • Position: vision • Orientation: inertial

42. Inside-Looking-OutOptical Tracking • The tracking device carries the camera that tracks markers in the environment. • Intersense Tracker • 3rdTech HiBall Tracker Images from: High-Performance Wide- Area Optical Tracking The HiBall Tracking System, Welch, et. al. 1999.

43. HiBall Tracker • Position • Pretty good • Orientation • Very good • Latency • LEPDs can operate at 1500 Hz Six Lateral Effect Photo Dioides (LEPDs) in HiBall. Think 6 cameras.

44. Angle Measurement Measurement of the bend of various joints in the user’s body Used for: • Reconstruction of the position of various body parts (hand, torso). • Measurement of the motion of the human body (medical) • Gestural Interfaces

45. Angle Measurement Technology • Optical Sensors • Have an emitter on one end and a receiver on the other. • As the sensor is bent, the amount of light that gets from the emitter to the receiver is attenuated in a way that is determined by the angle of the bend. • Examples: Flexible hollow tubes, optical fibers • VPL Data Glove

46. Angle Measurement Technology (cont.) • Strain Sensors • Measure the mechanical strain as the sensor is bent. • May be mechanical or electrical in nature. • Cyberglove (Virtual Technologies)

47. Joints and Cyberglove Sensors Proximal Inter- phalangeal Joint (PIP) Interphalangeal Joint (IP) Metacarpophalangeal Joint (MCP) Metacarpophalangeal Joint (MCP) Abduction Sensors Thumb Rotation Sensor

48. Cyberglove Accuracy

49. Cyberglove Accuracy (Adj.)

50. Angle Measurement Technology (cont.) • Exoskeletal Structures • Sensors which attach a rigid jointed structure to the body segments on either side of a joint. • As the joint bends, the angle between the body segments is measured via potentiometers or optical encoders in the joints of the exoskeleton. • Exos Dexterous Hand Master