Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection

1. Low-Cost Multi-Touch Sensing through Frustrated TotalInternal Reflection Jefferson Y. Han,New York University Presented by: Cody Boisclair

2. Introduction • Touch sensitivity is fairly common in electronics today… • …but only for a single point of contact at any given time. • Touch-sensitive devices that allow for multiple points of contact have thus far been expensive or difficult to produce.

3. Why multi-touch? • Allows a user to interact with a system with more than one finger at once • Chording • Operations using both hands • Also allows multiple users to interact on the same touch-sensitive platform • Interactive walls • Interactive tabletops

4. Prior approaches to the problem • Matrix of smaller sensors • Requires many connections, which severely limits resolution • Visually opaque - no touch-screen • Video cameras • Measuring of brightness • Stereo cameras • Identifying tracking markers in gel

5. A new approach… F T I R rustrated otal nternal eflection

6. Total Internal Reflection • When light encounters a medium with a lower index of refraction (e.g., going from glass to air), its refraction depends on the angle at which it hits the border. • Beyond a certain critical angle, light is not refracted, but instead reflects entirely within the material. • This is the basis for fiber optics and other optical wave guides.

7. Frustrated Total Internal Reflection • If another material touches that within which the light is reflecting, the reflection is frustrated, causing the light to escape. • This has been used in the past: • fingerprint imaging • early touch sensors (1970s!) • tactile sensors for robotic grips

8. A Schematic of FTIR (Kasday, 1984)

9. Using FTIR for touch sensitivity • A clear acrylic sheet is used as the touch surface. • 16 inches x 12 inches in prototype • Edges of surface lit by infra-red LEDs to produce total internal reflection. • A video camera is mounted under the surface and facing it. • When the surface is touched, the light escapes and registers on the camera.

10. Using FTIR for touch sensitivity • Basic image-processing techniques are performed on the camera output to identify the points of contact. • Computer-vision techniques are used to interpret the motion of contact points as discrete touches or strokes. • Processing easily handled in real-time by a 2 GHz Pentium IV processor

11. Advantages to this approach • High capture rate and resolution • 30 frames per second • 640x480 • True zero-force touch sensitivity • Inexpensive to construct • Scalable to much larger (even wall-sized!) surfaces • Transparent: can be combined with rear-projection display

12. How is projection achieved?

13. Disadvantages • Requires significant space behind touch surface for camera • Gloves, certain types of styluses, and even dry skin may not register • a function of refractive index • Residues on surface (e.g., sweat) also produce FTIR effect that may build up

14. Some applications… Video Clip at NYU (Local Copy) • Uses a 36x27-inch rear-projection screen. • Touch information is sent to programs using OSC (Open Sound Control) protocol

15. Any Questions? ?