Gesture based mobile training of intercultural Behavior

1. Gesture based mobile training of intercultural Behavior By SHASHANKA SUKKALA CSU ID-2571900

2. Outline • Introduction • Approach • Implementation • Discussion • Conclusion

3. introduction An approach to intercultural training of nonverbal behavior is presented that draws from research on role-plays with virtual agents. a mobile game is realized where the user acquires knowledge about German emblematic gestures and tries them out in role-plays with virtual agents. The paper presents the GAME approach along with details on the gesture recognition by making use of build-in acceleration sensors of smart phones. GAME brings together ideas from situated learning and intercultural training in an integrated approach and paves the way for new mobile-learning concepts.

4. Culture • What is Culture? The notion of culture itself is a multiply defined notion that gives rise to many misconceptions. Thus, it is necessary to specify exactly what is meant by culture in the envisioned training system as this notion affects several levels of the system like learning scenarios or the behavior of the virtual characters.

5. communication We live in globalized world and we have to make communication possible across different cultures and continents.

6. Communicating across cultures needs more than language • Some cultures • Need more background and context • Need different messages All cultures have different reactions to • Tone of voice • Nonverbal language Facial expressions , Style , Gestures All cultures have potential for miscommunication. The more sensitive the message the more miscommunication is.

7. Cultural differences The main problem with intercultural communication is cultural differences during interpersonal encounters In this paper we present an approach to intercultural communication by Virtual Agents which train them by giving proper coaching to the trainees about different cultures and behavior's

8. APPROACH: Virtual Agents Virtual agents are interactive systems that are capable of engaging in conversation with human being through speech, gestures , and facial expression Applications of virtual agents • Information presentation • Entertainment • Education

9. Hofstede’s Cultural Dimensions • Hofstede presents with his theory of cultural dimensions that defines culture as a five-dimensional concept and relates positions on the dimensions to certain behavioral pattern • Hierarchy • Identity • Gender • Uncertainty • Orientation

10. Hierarchy

11. Identity

12. Gender

13. uncertainity

14. orientation

15. Intercultural training • With G.A.M.E we aim at providing the means to train gestures anytime anywhere in role plays following suggestions from hofstede • Awareness • Knowledge • skill

16. Two dimensional model of intercultural coaching

17. Game approach: building blocks • Overview of GAME • GAME Architecture • Mobile gesture recognition • Evaluating GAME

18. G.A.M.E Gesture activated mobile edutainment Is an application in which the system integrates Interactive role plays with virtual character, which allows culture specific-gestures, making use of the sensor technology of smart phones. A mobile game is developed where user acquires knowledge about German emblematic gestures in role play with virtual agents

19. Overview of GAME • A mobile platform is chosen for coaching, that allows for contextual training session that is any time and anywhere using the htc based windows mobile • The user can choose to either run the game In single player mode Or Competitive mode

20. Game Archetecture

21. Users can play simultaneously together making one user as master and others as slaves. • The user can load new scenarios and gestures from the server Game feature two modes one dedicated to training specific skills , the other allowing the user to put new knowledge and skills to a test specific situation

22. Emblems • What are emblems? Emblems are special type of gestures • What are gestures? A motion of the limbs or body made to express or help express thought or to emphasize speech. Emblems are culture specific . • There are different set of emblems in different cultures • Same emblems can have different meaning

23. Mobile gesture Recognition • Mogle (mobile gesture learning environment) In mogle we use Acceleration sensors of handhelds(mobile) for training german emblematic gestures . Mogle restricts the number of features and offers only a naïve bayes classifier in order to reduce calculations efforts on mobile device , to make it leaner and faster to operate on mobile devices

24. To train the classifier • Training set is recorded for each gesture by different user • Then features are calculated on raw signal • Resulting vector along with information is used for training Mogle Windows mobile (htc diamond touch) Acceleration rate of sensors is 60 hz for each axis

25. Evaluating MoGLE • We evaluate MoGLE using German emblems by performing various gestures with mobile phone. • Each user was asked to take mobile phones and perform the gesture several times • The information gathered from these tests are used to create a database of training sample for classifier • Recognition rates for the training sample are calculated by a 14 fold cross validation technique

26. Authoring of learning units • This section deals how scenarios are defined and according to user inputs and cut scenes are generated • A greeting scenario is taken with a trigger video showing an agent waiting in the garden user interacts with the agent i.e. come here gesture, resulting in a cut scene Depending on the performance of the user one of three states are activated

27. If the performance of user is real bad , i.e the come here gesture was not recognized the system remains in the state of agent waiting. If the performance is good and gesture is recognized the system moves into the state, ‘Agent moves close to the user’ and corresponding cut scene is played. If the user performance is good and gesture is recognized , then next trigger video is played and again based on the performance of the user cut scene is played.

28. The GAME: APPLICATION THE APPLICATION CONSITS OF TWO MODES • TRAINING MODE: Concentrated on specific gestures • GAME MODE: Check his knowledge skills

29. The start screen offers three options. • Gesture training(default option) • Quick training • Random training GAME MODE We realize that experience based role play is standard technique for intercultural training. For example the scenario of garden meeting. How the user applies his verbal and nonverbal skills of the target culture and comes up with right behavior that of target Evaluating Game Two different types of evaluations are conducted • Usability Evaluation. • Evaluation of training effect.

30. Usability Evaluation : Focused on general usability of system , i.e in order to show, the interface and game are attractive to users or not. Evaluation is conducted by taking 20 participants , which consists of training phase followed by a single player game, and at the end participants are requested to fill out AttrakDiff questionnaire, which measures the hedonistic and pragmatic qualities of the system. 1)LOG DATA: All user actions will be logged during training and role play to analyze the success of gesture executions. 2)Hedonistic and Pragmatic qualities: By requesting a graded response to adjective pairs like ‘‘complicated–simple’’, the AttrakDiff questionnaire results in a rating of the product’s quality 3)Subjective impressions: Participants have been asked to write down their subjective impressions about the game play

31. Evaluati:on of Cultural Training :In this section we take implicit culture background of the participants as an experiment condition. The study is done in two steps • Skill training • Performance rating Skill Training: Participants from other cultures than the target culture interact with the system (test group) or learn about German emblematic gestures in a traditional way (control group). Performance Rating :Video recordings of gestures from the two groups are rated by participants from the target culture based on their implicit knowledge about good performance of the gestures Test group vs Control group TG outperforms CG, i.e. performance ratings from native speakers are better for the test group and thus the experience based training results in a positive effect

32. RESULT Results from Table give the mean rating for each gesture Of the test and control groups. At test was run on the data and reveals high differences for three of the gestures (Come Here, Stupid, Go On) , with higher ratings for the test group.

33. Conclusion The work presented in this paper is based on the idea of training users with the help of gestures using mobile technology for intercultural behavior. We present GAME. A mobile edutainment platform , that challenges the user with active tasks where he has to put his knowledge and skills about non-verbal behavior to test in interactions with virtual characters. Two evaluation studies have been presented that show the positive potential of this approach and high-light the fact that the experience-based gesture training outperforms traditional methods. The experience-based role-plays with virtual characters have been brought to the mobile device, freeing the user from desktop-based stationary interactions