A Virtual Keyboard with Multi Modal Access for people with disabilities

1. A Virtual Keyboard with Multi Modal Access for people with disabilities Vijit Prabhu1, Girijesh Prasad2 1 Computer Science & Engineering, Indian School of Mines, Dhanbad, Jharkhand -826004, India 2Intelligent Systems Research Centre (ISRC), University of Ulster, Magee Campus, Derry, N. Ireland, UK E-mail: g.prasad@ulster.ac.uk • Introduction • Persons with speech and motor disorders face problems in expressing themselves in an easy and intelligible way. • Computer based augmentative and alternate communication (AAC) systems are developed to assist them. • A Virtual Keyboard (VK), also called as On Screen Assistive Keyboard is a commonly used AAC system. • A VK is characterized by the input modalities and the layout. Background • “Hex-o-Spell” [1] virtual keyboard is an EEG based Brain Computer Interface that uses machine learning techniques to identify brain signals. • Dasher[2] is an information-efficient text-entry interface, driven by natural continuous pointing gestures via a access switch, touch screen, or eye-tracker. Figure 1: Hex-o-Spell Virtual Keyboard We revisited the problem to incorporate a number of input modalities and optimize the design layout to achieve optimum performance from the Virtual Keyboard. The Virtual Keyboard • 1. Working Model • Figures2(a) and 2(b) show the working of the Virtual Keyboard. • The pointer can be rotated either by the user or it can be rotated at a fixed speed by the computer itself. • The pointer rotated till it reaches the desired sub circle. • A trigger is used to select the sub circle which expands [figure 2(b)] • The pointer is again rotated to point to the desired character • The trigger is used to select the character and it is typed in to the corresponding application. Figure 2(b) Expanded Sub Circle: pointer points at the character L Figure 2 (a) Virtual Keyboard: pointer pointing at the sub circle • 3. Layout Design • Different characters have different frequency of occurrence in English Text. [3] [4] • Different positions of characters in the layout , require different amount of access activity. • Design Principle- higher the frequency of occurrence of character, lower should be the amount of activity required to access it. • Clashes in positioning of character were resolved based on the probability of the character being the first letter of the word. [5] • 2. Modes of Access • Any one or the combination of three incorporated technologies can be used for access. • The Brain Computer Interfacing (BCI) based on EEG uses ‘vivid imagination’ of a motor activity as a trigger. • The Eye Tracker technology uses prolonged gaze and/or eye blink as a trigger. • Access switches (also called as soft switches) use any active body part such as hand, foot, mouth or head as a trigger Figure 2 (c) : Subject using Eye tracker as mode of access. Analysis and Results Conclusion xi: access activity required (including rotation and selection) to access the ithcharacter p(xi): probability distribution function based on relative frequency of occurrence in text in [3][4] The expected value [6] or mean of access activity required per character is given by: E(x)=∑ (xi. p(xi)) A similar VK with ‘alphabetical ordering’ has expected activity per character = 6.55 Our layout has the expected activity per character = 4.55 The analysis confirms that our layout has better performance than alphabetical ordering as in [1] The multiple modes of access allows same the VK to be used by a wider spectrum of people with different levels of speech and motor disorders. Figure 3 : Comparison of our layout against the alphabetical ordering Position (m,n) refers to mth sub-circle; nth location in the sub-circle References [1] Blankertz et al., The Berlin Brain-Computer Interface Presents The Novel Mental Typewriter Hex-O-Spell, In: Proc. of the 3rd International Brain-Computer Interface Workshop and Training Course 2006, September 2 1-24 2006, Verlag der Technischen Universität Graz.. [2] Dasher-Information-Efficient Text Entry-Hanna Wallach, University of Cambridge/University of Pennsylvania http://www.mendeley.com/research/dasher-an-efficient-keyboard-alternative/ [[3] Lewand, Robert (2000). Cryptological Mathematics. The Mathematical Association of America. p. 36. ISBN 978-0883857199 [4] Lee, E. Stewart; Essays about Computer Security; University of Cambridge Computer Laboratory, p. 181 [5] Letter Frequency, Wikipedia, http://en.wikipedia.org/wiki/Letter_frequency [6] Expected Value, Wikipedia, http://en.wikipedia.org/wiki/Expected_value