vSmiley : The imaging of emotions through vibration patterns.

1. AAFG 2005 vSmiley : The imaging of emotions through vibration patterns. Deepa Mathew Department of Computer Sciences University of Tampere, Finland deepa.mathew@uta.fi April, 2005

2. vSmiley INTRODUCTION Tactons are becoming a popular widget in various research programs to build new multi-model or multi-tasking user-interfaces, mainly because of it’s ability to directly interpret with human skin (cutaneous), which in turn gives an impression of ‘sense of feel’. Various tactile patterns can be designed in a way, even to express emotion with sense of feel when it is presented as Smiley or in any other forms like icons, graphics and so on. Using tactons can even be a better option when building an user-interface for deaf/blind and also deaf-blind users In the game “vSmiley” we have used the vibrating patterns (Tactons) for imaging of emotions. And this game is also intended for deaf or visually impaired children, using vibrations (Tactons) as the common root. Deepa.Mathew p 02_18 18.04.2005

3. vSmiley • GOAL • The goal of the project vSmileys is an empirical study of the possibility to substitute emoticons by vibro-tactile patterns. • Tactons could be a usesful means of communicating information in user interfaces for the visually challenged and people with hearing disabilities.So,the game could be a starting point to get familiarised with the sematic patterns. Deepa.Mathew p 03_18 18.04.2005

4. vSmiley METHOD • Similar to the envelope of sound patterns (earcons) mechanical vibrations can be composed into the vibro-tactile patterns (tactons). • Some earcons were produced and widely used to signify a semantically-completed messages and feedback cues in GUI. Vibro-tactile patterns are employed as a way of communication especially for and between people with sensorial deficit (deaf, blind, deaf-blind). Figure1: Sound wave envelope Deepa.Mathew p 04_18 18.04.2005

5. vSmiley TECHNIQUES The tactons were designed for 9 smileys signifying different emotions. The aim of the pilot testing through the game was estimation of relative identification of the semantics of the constructed tactons (vSmileys) by comparing them with the emotion of the smileys presented visually. Basic vibration patterns (Figure 2) have been used and combined to get the desired sample (vSmiley) which should contain semantics similar to the graphical prototype (smiley). Basic patterns can be combined to create composite vibro-tactile messages. Figure2: Vibrating patters used in designing vSmileys, Immersion Studio 4.0 Deepa.Mathew p 05_18 18.04.2005

6. vSmiley Apparatus and Procedure Software: ‘Immersion Studio 4.0’ was used in designing vSlileys to transform smileys into composite vibro-tactile patterns. Hardware: An Logitech iFeel optical mouse (Figure 4) connected via USB port was used to display the vibro-tactile patterns. Figure 3: Logitech iFeel optical mouse Deepa.Mathew p 06_18 18.04.2005

7. vSmiley There are some other devices as well that could be used. For example: Engineering Acoustics C2 Tactor (Figure 4) and TACTAID VBW32 transducer (Figure 5) These devices was used in the research done by Brown et al [1] Figure 5: TACTAID VBW32 transducer. Figure 4: Engineering Acoustics C2 Tactor [2] http://www.dcs.gla.ac.uk/~stephen/papers/WorldHaptics_Brown.pdf Figure 6: C2 Tactor attached to finger. Deepa.Mathew p 07_18 18.04.2005

8. vSmiley DESIGNING THE TACTONS Before vSmileys have been created, the appropriate parameters for vibro-tactile patterns in which information can be encoded must be identified [1]. The most obvious parameters to use in vSmiley construction are the basic parameters of basic vibratory patterns such as frequency, amplitude, waveform (envelope: attack, sustain, decay) and duration of each segment/burst. However, in designing the vSmileys have been used only the frequency and duration, these parameters were changed to encode 9 smileys. Semantic information could be encoded by manipulating the duration of pulses. While duration alone could be used as a parameter, combining pulses of different durations to form rhythms would offer more flexibility [1]. Deepa.Mathew p 08_18 18.04.2005

9. vSmiley TESTING PROCEDURE Five participants, (2 females and 3 males) with normal vision and hearing took part in the pilot testing of the vSmileys game. None of the participant had any previous experience in this kind of testing and especially had no idea about tactons. They are in the age group of 23-28 (2 students and 3 employees) Five minutes training time was given to each of them to memories vSmileys. If needed, then extra time was given. After the training time was over the participants where asked to identify different vSmileys by feeling the vibratory patterns prsented with Logitech iFeel optical mouse with built-in shaking motor. The test consisted of 90 trials (9 vSmileys x 10 times) and the tactons where selected randomly. Data were recorded on the recognition of the correct smiley and stored in a log file after completion the game. Deepa.Mathew p 09_18 18.04.2005

10. vSmiley • The game is played as explained below • Click on each smiley as many times as needed to memories the vSmileys (vibro-tactile patterns). • After player have memorised all the patterns, s/he can activate the testing phase by tapping the space bar. • Tapping the spacebar produces the test-pattern. Player can repeat this vSmiley as many times as needed. • The player has to identify the test-pattern and point by click the correct smiley having the same meaning. Figure7: vSmiley game Deepa.Mathew p 10_18 18.04.2005

11. vSmiley RESULTS AND DISCUSSIONS The players had some trouble choosing the right smiley in the beginning but the results were gradually improved. And they were able to distinguishthe vibrating patterns and identify the correct smiley after few trials. But they had some trouble when identifying the somewhat-similar vSmileys. They got easily confused and made more errors. Deepa.Mathew p 11_18 18.04.2005

12. vSmiley The average number of repeatitions per vSmiley in training phase for each player. Each of the players had little difficulty in memorising the 6th and 9th vSmileys. Maybe because of its similarity. Deepa.Mathew p 12_18 18.04.2005

13. vSmiley The average number of repeatitions per vSmiley in testing phase for each player. Deepa.Mathew p 13_18 18.04.2005

14. vSmiley The average number of wrong trials per vSmiley. Deepa.Mathew p 14_18 18.04.2005

15. vSmiley The average number of repetitions per image throughut the test for all the players. Deepa.Mathew p 15_18 18.04.2005

16. vSmiley The average error rate during the test. Some vSmileys were easily distinguished and some were not. Deepa.Mathew p 16_18 18.04.2005

17. vSmiley CONCLUSIONS Vibro-tactile patterns and semantic sequences can play a vital role for building new multi-model or multi-taskinguser-interfaces especially for users having a sensorial deficit and also introduce a possible alternative way of communication. Tactons were already been used in areas such as tele-operation or displays for blind people to provide sensory substitution [2]. But not to its true abilities. More research should be done to know the true usefulness of the tactile patterns. And based on the pilot experiment it is obvious that it is easy to identify and differentiate different vibrating patterns. And with proper research and knowledge, tactons could replace speech or short speech messages which could be vital for the deaf users. Deepa.Mathew p 17_18 18.04.2005

18. vSmiley References [1] Lorna M. Brown, Stephen A. Brewster and Helen C. Purchase. A First Investigation into the Effectiveness of Tactons. Glasgow Interactive Systems Group, Department of Computing Science, University of Glasgow, UK [2] Dr Steven Wall, An investigation of multimodal interaction with tactile displays. Department of Computing Science, University of Glasgow. Click to go the first slide Deepa.Mathew p 18_18 18.04.2005

19. Thank you!Any Questions?