1 / 15

Alexandre Cadiou ISEP, Paris, France alexandrediou@isep.fr isep.fr

Hands-Free Mouse Control System for Handicapped Operators ICANDO. Alexandre Cadiou ISEP, Paris, France alexandre.cadiou@isep.fr www.isep.fr. Alexey Karpov SPIIRAS, Saint-Petersburg, Russia karpov@iias.spb.su www.spiiras.nw.ru/speech. 28 June 2006, SPECOM 2006, Saint - Petersburg, Russia.

lala
Download Presentation

Alexandre Cadiou ISEP, Paris, France alexandrediou@isep.fr isep.fr

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hands-Free Mouse Control System for Handicapped Operators ICANDO Alexandre Cadiou ISEP, Paris, France alexandre.cadiou@isep.fr www.isep.fr Alexey Karpov SPIIRAS, Saint-Petersburg, Russia karpov@iias.spb.su www.spiiras.nw.ru/speech 28 June 2006, SPECOM 2006, Saint - Petersburg, Russia

  2. Contents • ICANDO: Intellectual Computer Assistant for Disabled Operators • Actuality • Modification of head tracking modul of ICANDO • The multiplication of tracking points • Restoration method of the lost tracking points • Method of cursor velocity adaptation • Experiment • Video of full system

  3. Actuality Assistance to persons without hands or with disabilities of their hands or arms for human- computer interaction • Recover a social life • Reach a good position • To be less assisted

  4. General architecture of the system Head movements Human’s speech Users who have difficulties using standard devices could manipulate screen cursor merely by moving the head and giving the speech commands instead of clicking the buttons. ASR engine SIRIUS performs the RUSSIAN speech recognition Speech recognition Synchro- nization Head tracking Speech command Mouse Coordinates (x,y) Multimodal fusion Multimodal command

  5. Table of main voice commands

  6. Head Tracking System (new version) Intel OpenCV library (Open Source Computer Vision Library) 2 stages of system work: • calibration (Haar-based face detection, capturing the position of tip of nose, upper lip, middle of the eyes and eyes) • tracking (iterative Lucas-Kanade algorithm for optical flow to track the position of user’s face) Real time mode

  7. Multiplication of tracking points • 5 tracking points positioned on two lines • Customization of positions of tracking points • Increase of robustness of ICANDO

  8. Restoration method of the lost tracking points • Rectangles define operation areas • Restoration of the lost tracking points in their corresponding operation areas X Y

  9. Restoration method of the lost tracking points • Definition of reference tracking point(red point) • Stability of the head tracking system X Y

  10. Needs of cursor velocity adaptation • We need to adapt the cursor velocity to the needs of disabled operators • Freedom of movements limited for disabled operators • Problem of stability of mouse cursor with only one velocity for precise work

  11. Method of velocity adaptation • This method is based on the velocity measurement of the movements of operator’s head. • This method measures the number of pixels traversed by the point 2 tracked by the webcam between two consecutive frames.

  12. Experimental results • The system was tested on the task of drawing a picture with the MS Paint editor. • To compare both versions of the ICANDO device, a complex picture has been drawn.

  13. The comparison of two systemversions

  14. Video of full system

  15. Thank you! QUESTIONS? SPIIRAS, SAINT PETERSBURG, RUSSIA ISEP, PARIS,FRANCE

More Related