1 / 17

Facilitating User Interaction with Complex Systems via Hand Gesture Recognition

Explore how gesture recognition can improve user interaction with complex systems, replacing traditional input methods. The proposed approach outlines the system architecture, implementation details, and future directions to optimize user experience and interface extensions.

anitagordon
Download Presentation

Facilitating User Interaction with Complex Systems via Hand Gesture Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Facilitating User Interaction with Complex Systems viaHand Gesture Recognition MCIS Department Knowledge Systems Laboratory Jacksonville State University Joshua R. New, Erion Hasanbelliu, and Mario Aguilar

  2. Outline • Motivation • System Architecture • Implementation Overview • Proposed Approach • Demonstration • Future Directions

  3. Motivation • Gesturing is a natural form of communication • Interaction problems with the mouse • Have to locate cursor • Hard for some to control (Parkinsons or people on a train) • Limited forms of input from the mouse

  4. Motivation (2) • Interaction Problems with the Virtual Reality Glove • Reliability • Always connected • Encumbrance

  5. Gesture Recognition System System Architecture User Rendering Update Object User Interface Display Hand Movement Image Capture Image Input Standard Web Camera

  6. Implementation Overview • System: • 1.6 Ghz AMD Athlon • OpenCV and IPL libraries (from Intel) • Input: • 640x480 video image • Hand calibration measure • Output: • Rough estimate of centroid • Refined estimate of centroid • Number of fingers being held up • Manipulation of 3D skull in QT interface in response to gesturing

  7. Implementation Overview (2) • Hand Calibration Measure: • Max hand size in x and y orientations in # of pixels

  8. Implementation Overview (3) Saturation Channel Extraction (HSL space): Original Image Hue Lightness Saturation

  9. Extract Saturation Channel Threshold Saturation Channel Find Largest Connected Contour Proposed Approach

  10. Segment Hand From Arm Calculate Refined Centroid Calculate Centroid Proposed Approach (2)

  11. Count Number of Fingers Proposed Approach (3) • The finger-finding function sweeps out a circle around the rCoM, counting the number of white and black pixels as it progresses • A finger is defined to be any 10+ white pixels separated by 17+ black pixels (salt/pepper tolerance) • Total fingers is number of fingers minus 1 for the hand itself

  12. Proposed Approach (4) • System Runtime: • Current time – 41 ms for one image from camera • Processing Capability on 1.6 Ghz Athlon: • 24 fps

  13. Demonstration System Configuration System GUI Layout

  14. Demonstration (2) Gesture to Interaction Mapping Number of Fingers: 2 – Roll Left 3 – Roll Right 4 – Zoom In 5 – Zoom Out

  15. Demonstration (3)

  16. Demonstration (4)

  17. Future Directions • Optimization • Calibration Phase • Defining Hand Orientation • Learning System • Interface Extensions For additional information, please visithttp://ksl.jsu.edu.

More Related