1 / 18

Gesture Recognition / Sign Language

Gesture Recognition / Sign Language. Lukas Bloder Johannes Bannhofer SE09 MUS2 SS10. Overview. Hardware Sign Language Live Demo System Architecture System Tools Technologies Problems Fazit. Hardware. P5 Glove API: http://www.robotgroup.org/index.cgi/P5Glove. CyberGlove :

onan
Download Presentation

Gesture Recognition / Sign Language

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Gesture Recognition / Sign Language Lukas Bloder Johannes Bannhofer SE09 MUS2 SS10

  2. Overview • Hardware • SignLanguage • Live Demo • System Architecture • System Tools • Technologies • Problems • Fazit

  3. Hardware P5 Glove API: http://www.robotgroup.org/index.cgi/P5Glove CyberGlove: http://www.immersion.com http://www.golem.de/0512/42086.html MIT Color Glove Handtracking http://people.csail.mit.edu/rywang/handtracking/

  4. Hardware CyberGlove :

  5. Hardware • Number of sensors: 18 or 22 • Sensor Resolution: 0.5 degrees (typical) • Sensor Data Rate: 90 records/sec minimum (100 records/sec typical). • Operating system andhosts: Windows 2000 and XP • Operating Range: 30 ft radius from USB port • Interface: USB port for the wireless receiver CyberGlove II:

  6. Sign Language • American manual alphabet

  7. Sign Language • Substitution signs • Dynamic signs: J, Z • Additional Signs • Space, enter, delete, variouscommands

  8. Demo Time!

  9. System • C++ API (Partiallyfrom original sourceof 1998) • JNI Bridge • Application: • Exchangeable Processing (Matlab, weka) • Rules (substitutionsigns, comamnds) • Clients (Commandline, TTS, Graphical)

  10. System Architecture

  11. Classification using ANN • Matlabnntool

  12. Classification using ANN • Matlab – Erros recognizing letters

  13. Processing Rules • Rules to process more complex signs • Recognition splitted to Wrist/Fingers • Evaluation with rules

  14. System Tools • Data Collector • Data Aggregator

  15. Technologies Used • C++ / Java • Matlab • MaryTTS

  16. Problems • Old API • Matlab /generating JAR Files • API license problems • Training data • Inconsistent sensor data

  17. Fazit • Old Hardware still does the job • Don’t touch machine generated code • Generating good training data -> hard work

  18. Thanks for your attention! Lukas Bloder Johannes Bannhofer SE09 PEG SS10

More Related