1 / 17

6.870 Final Project Webnnel: A channel-based Web navigation system

6.870 Final Project Webnnel: A channel-based Web navigation system. Chen-Hsiang Yu and Oshani Seneviratne {chyu,oshani}@mit.edu. Outline. Introduction (Jones) Motivations (Jones) Related Work Web automation and customization (Jones) Speech recognition (Oshani) Our Approach

oriana
Download Presentation

6.870 Final Project Webnnel: A channel-based Web navigation system

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 6.870 Final Project Webnnel: A channel-based Web navigation system Chen-Hsiang Yu and Oshani Seneviratne {chyu,oshani}@mit.edu 6.870 Multimodal User Interface

  2. Outline • Introduction (Jones) • Motivations (Jones) • Related Work • Web automation and customization (Jones) • Speech recognition (Oshani) • Our Approach • Web customization and automation (Jones) • Speech recognition (Oshani) • The integration of command extension with speech recognizer (Jones, Oshani) • Demonstration • Challenges (Oshani) & Future Work (Jones) • Discussion (Jones) • References 6.870 Multimodal User Interface

  3. Introduction • I 6.870 Multimodal User Interface

  4. Motivations • I 6.870 Multimodal User Interface

  5. Related Work • Web Automation and Customization • Point 1 • Point 2 • Speech Recognition • Microsoft Vista Speech Recognition Engine • Apple Mac Speech Recognition Engine (But none of the above provide the level of customization offered by Webnnel!) 6.870 Multimodal User Interface

  6. Our Approach - Web Customization and Automation • I 6.870 Multimodal User Interface

  7. Our Approach - Speech Recognition Used the Mac OS Speech Recognition Engine Written in Objective C Highly flexible To add new commands you have to… Allocate and initialize an instance of NSSpeechRecognizer. Set the commands that the object should listen for using the setCommands: method. Set a delegate for the NSSpeechRecognizer object that implements the speechRecognizer:didRecognizeCommand May 14, 2008 6.870 Multimodal User Interface 7

  8. Our Approach - Integration Web Customization and Automation: Apple Scripts: Acts as the “glue” between the speech recognition and the Webnnel Firefox Extension Custom scripts for each speech command Perform keystrokes at the Webnnel command prompt upon recognition May 14, 2008 6.870 Multimodal User Interface 8

  9. Demonstration • I 6.870 Multimodal User Interface

  10. User Study Conducted a qualitative study on 4 users Asked the users to perform 2 tasks using the Webnnel speech recognition system Task 1: Go to a certain website Task 2: Go to their web-based email system May 14, 2008 6.870 Multimodal User Interface 10

  11. User Study (cont) • Recognition Accuracy (from the 16 commands we asked them to test the system with): 6.870 Multimodal User Interface

  12. User Study (cont) General Comments from the users: Commands are natural and easy to remember Liked the tag system Shorter the command it’s better There should be ways to enter the URL directly in to the address bar as well May 14, 2008 6.870 Multimodal User Interface 12

  13. Challenges • Early experimentation on CMU-Sphinx4 Java based speech recognition failed • Too many configuration parameters to consider • Our custom language model and grammar had a very poor recognition accuracy • Achieving cross platform compatibility: • Compared to the Mac OS, Windows (XP, Vista) and Linux (Ubuntu 7.10) did not have good support for speech recognition. • The quality of the microphones vary across different computers • Introducing many speech commands generally lowers the accuracy of the entire system • Having a stress ball around was very handy while testing the speech recognition :) 6.870 Multimodal User Interface

  14. Future Work Porting the speech recognition aspect of the Webnnel system to other platforms. May 14, 2008 6.870 Multimodal User Interface 14

  15. References • Avot mV, http://www.avotmedia.com/ • Bigham, J. P., and Ladner, R. E. Accessmonkey: a collaborative scripting framework for web users and developers. In W4A '07, ACM Press, pp. 25-34, 2007. • Bolin, M., Webber, M., Rha, P., Wilson, T. and Miller, R.C. Automation and customization of rendered web pages, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005. • Apple Speech Recognition Engine, http://developer.apple.com/documentation/Cocoa/Conceptual/Speech/Articles/RecognizeSpeech.html • CMU-Sphinx Speech Recognition Engine, http://cmusphinx.sourceforge.net/html/cmusphinx.php • Greasemonkey, https://addons.mozilla.org/en-US/firefox/addon/748 • Joost, http://www.joost.com/ • Microsoft Windows Vista Speech Recognition system http://www.microsoft.com/enable/products/windowsvista/speech.aspx • Mogulus, http://www.mogulus.com/ • Petrie, H., Hamilton, F. and King, N. Tension, what tension? Website accessibility and visual design. Proceedings of the 2004 international cross-disciplinaryworkshop on Web accessibility (W4A), pp. 13-18, 2004. 6.870 Multimodal User Interface

  16. References (cont.) • Richards, J. and Hanson, V. Web accessibility: a broader view. Proceedings of the 13th international conference on World Wide Web, pp. 72-79, 2004. 6.870 Multimodal User Interface

  17. Any Questions? {chyu,oshani}@mit.edu May 14, 2008 6.870 Multimodal User Interface 17

More Related