1 / 10

Applications of Image Processing to Sign Language Recognition

Applications of Image Processing to Sign Language Recognition. Overview. Average person typing speed Composing: 19 wpm Transcribing: 33 wpm Sign speaker Full sign language: 200—220 wpm Spelling: est. 50 wpm. Purpose and Scope. Native signers faster “typing” Benefits:

adolph
Download Presentation

Applications of Image Processing to Sign Language Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Byron Hood Computer Systems Lab Project 2007-08 Applications of Image Processing to Sign Language Recognition

  2. Byron Hood Computer Systems Lab Project 2007-08 Overview • Average person typing speed • Composing: 19 wpm • Transcribing: 33 wpm • Sign speaker • Full sign language: 200—220 wpm • Spelling: est. 50 wpm

  3. Byron Hood Computer Systems Lab Project 2007-08 Purpose and Scope • Native signers faster “typing” • Benefits: • Hearing & speaking disabled • Sign interpreters • Just letters & numbers

  4. Byron Hood Computer Systems Lab Project 2007-08 Research • Original • Related projects • Using mechanical gloves • Tracking body parts • Image techniques • Edge detection • Line detection

  5. Byron Hood Computer Systems Lab Project 2007-08 Program Architecture (1)‏ • Webcam interface • Using Video 4 Linux 1 • Saves images to disk • Edge detection • Robert’s Cross • Line detection • Hough transform

  6. Byron Hood Computer Systems Lab Project 2007-08 Program Architecture (2)‏ • Line processing • Image data dropped • AI to find “best fit” • Letter matching • Load hand data from XML files • Try to match hand

  7. Byron Hood Computer Systems Lab Project 2007-08 Sample Code • From edge_detect.c : int row,col; // loop through the image for(row=0; row<rows; row++) { for(col=0; col<cols; col++) { // avoid the problems from trying to calculate on an edge if(row==0 || row==rows-1 || col==0 || col==cols-1) { image2[row][col]=0; // so set value to 0 } else { int left = image1[row][col-1]; // some variables int right = image1[row][col+1]; // to make the final int top = image1[row-1][col]; // equation seem a int bottom = image1[row+1][col]; // little bit neater image2[row][col]=(int)(sqrt(pow(left – right , 2) + pow(top - bottom, 2))); } } }

  8. Byron Hood Computer Systems Lab Project 2007-08 Testing (1)‏ • General testing model bhood@bulusan:~/syslab-tech$ \ > ./main images/hand.pgm in.pgm [DEBUG] Edge detect time: 486 ms [DEBUG] Wrote image file to `in.pgm’ Errors:0Warnings:0

  9. Byron Hood Computer Systems Lab Project 2007-08 Testing (2)‏ • Results: • Of edge detection • Similar for cropping or line finding • Automated testing difficult and impractical Original image (800 x 703)‏ Final image (800 x 703)‏

  10. Byron Hood Computer Systems Lab Project 2007-08 The Mysterious Future • Finish line detection • Detection and parameterization • Build AI to interpret lines • Finish camera-computer interaction • OPTIMIZE!!!

More Related