Gesture Analysis

1. Gesture Analysis Student Name : Saif Samaan Student Number :42083850 Supervisor: Manolya Kavakli

2. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

3. Abstract • A computer contains three main parts, input interfaces, processing devices and output interfaces. • Input and output interfaces called, Human Computer Interfaces (HCI), these interfaces provides the communication environment between human and computers.

4. Abstract • This project aims to explore human hand-gesture classification from a designer point of view, to support a discovery project sponsored by the Australian Research Council. • The discovery project examines a novel environment in which a designer can define the contour of a sketch by controlling a pointer using a pair of data gloves and can interact with the design product in 3D space.

5. Abstract • I provide an analysis study for current and previous hand-gesture recognition systems, also I experiment the gesture design process with twenty volunteers, and I defined a set of suggested gestures, techniques and possible future challenges.

6. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

7. Introduction • This section presents the background and a general description for the problem and the aim of this project. • Background • Human-computer interfaces (HCI) logic have developed through different stages using different tools, traditional tools like keyboards for text input, mice,2D and 3D graphical interfaces and sophisticated 3D virtual environment (VE) systems.

8. Introduction • Problem Description • Most human-machine interfaces based on joysticks, keyboards, and keypads, but few such interfaces use gestures. • Interaction based on hand gestures commonly uses two types of interface, gloved-based and vision-based .

9. Introduction • Vision-based interfaces require powerful image processing algorithms to segment the hand from stationary background and lighting conditions that enhance gesture classification accuracy. • A glove- based system requires the user to be connected to the computer. Even wireless systems currently offered in the market require the user to wear a glove. Furthermore, accurate devices are expensive and hard to calibrate.

10. Introduction • The Aim of this work • The main aim of this report is to explore human computer interaction and gesture recognition, in order to support the design process.

11. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

12. Literature Review 1-Video Capture(Vision-based Technology) • Using one camera or more to record images of human hand Gestures. • Infrared-camera Based System • Mono-camera Based Gesture Recognition System • Multi-camera Based Gesture Recognition System

13. Literature Review Multi-camera Mono-camera • Infrared camera

14. Literature Review 2-Image Segmentation • Threshold-Based Segmentation • Active Contour Models • Split and Merge • Boundary Based Segmentation

15. Literature Review 3-Image Processing The computer detects images produced from original hand image but usually the quality of the detected images comes along with a level of noise.

16. Literature Review Image capture Hand gesture detection using openCV

17. Literature Review //Code of capturing image sequence • int width=320;//Define the image resolution ratio • int height=240; • IplImage *pRgb=cvCreateImage(cvSize(width,height), IPL_DEPTH_8U, 3); videoInput video;//Create capturing object video.setupDevice(0, width, height);//Configure device while(1) { if(video.isFrameNew(0)  {  video.getPixels(0,(unsigned char *)pRgb->imageData, false, true);//Capture one frame  char c=cvWaitKey(1); if(c==27) break;//Press ESC to exit cvShowImage(“Video”, pRgb);//Show display window }

18. Literature Review Cyber-glove (http://www.immersion.com/ )

19. Literature Review • Vision-Based and Sensors-Based “Rise of the Planet of the Apes (2011)”

20. Literature Review 4-Gesture Classification David McNeill specifies four types of gestures, which are part of human communication, they are, iconics, metaphorics , beats and deictics.

21. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

22. Experiments and Analysis Twenty volunteers helped me in my experiments. J. Liu and M. Kavakli, “Temporal Relation between Speech and Co-verbal Iconic Gestures in Multimodal Interface Design,”

23. Experiments and Analysis • Differences and similarities in human gestures • Single Line description • 13/20 volunteers:

24. Experiments and Analysis • Differences and similarities in human gestures • Single Line description • 1/20 volunteer:

25. Experiments and Analysis • Differences and similarities in human gestures • Single Line description • 6/20 volunteer:

26. Experiments and Analysis Two Parallels Lines description .

27. Experiments and Analysis Twenty volunteers helped me in my experiments.

28. Experiments and Analysis Curve description.

29. Experiments and Analysis Circle description.

30. Experiments and Analysis Start and end point. First hand Second hand

31. Experiments and Analysis Critical region problem. First hand Second hand

32. Experiments and Analysis Curve description:

33. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

34. Survey and results • Left Public for future study • https://docs.google.com/spreadsheet/viewform?formkey=dFVOcklIaXV1b3c0d25BaHhpMVp5Q3c6MQ

35. Survey and results • 5-Do you agree that gesture-recognition system replace the keyboard and mouse? • 80% Not now • 17% Disagree • 3% Agree

36. Survey and results 9- If you have to chose only one input device with gesture recognition system, which one you will select? 80% Touch screen 15% Mouse 5% Keyboard

37. Report Outline • Abstract • Introduction • Literature Review • Experiments and Analysis • Experiments Survey and Results • Conclusion

38. Conclusion • The project is outlining a hand gesture recognition system that designed to be capable to recognize the hand gestures in real time, this will be in a picture of examining environment in which a designer can sketch by controlling a pointer using a pair of cyber gloves and can interact with the design product in 3D space.