Speech-Enabled .NET Framework Application for CIMS

1. Speech-Enabled .NET Framework Application for CIMS Murat Semerci Çağdaş Kayra Akman 15.06.2004

2. Project Goal Development of speech-enabled voice-only version of Electrical and Electronic Engineering Department’s Course Information Management System (CIMS)which is accessed via telephone network.

3. Development Environment • Microsoft Speech Application SDK Beta 4 • Microsoft Visual Studio .NET 2003 Enterprise Architect • Microsoft SQL Server 2000 • Windows 2000 Professional Edition SP 4 (Intel Pentium III 800 MHz, 256 MB RAM)

4. Overview SR Engine Prompts Database SERVER CLIENT USER TTS Engine SQL Server

5. Basic Dialog Flow START User Name Password Intelligent Mode ? YES NO Course Code/Course Name and Menu Course Code YES Menu AnotherQuery ? END

6. Error Handling • No Recognition: no speech detected or recognized phrase not covered by grammar • Invalid Input: recognized input does not match with user’s data (UserName, Password Course ID, Menu) • No Database Connection

7. SALT • Speech Application Language Tags: extension of HTML and XML to support speech-enabled web applications • Generated by Speech Application SDK • Manipulated directly or via C# codebehind pages • Supports client-side scripts to handle recognition or TTS events or prompts

8. Grammars • XML codes complying to W3C’s SRGS (Speech Recognition Grammar Specification) • Used by SR engine on the client side. • Results returned to server using W3C’s SML (Semantic Mark-up Language)

9. Grammars

10. Grammars <rule id="UserName" scope="public"> <one-of> <item> <?MS_Grammar_Editor GroupWrap?> <one-of> <item> <?MS_Grammar_Editor GroupWrap?> <ruleref uri="Library.grxml#Digit7" type="application/srgs+xml"/> </item> <item> <?MS_Grammar_Editor GroupWrap?> <ruleref uri="Library.grxml#Digit10" type="application/srgs+xml"/> </item> </one-of> <tag>$.Username = $.Username ? $.Username : {}; $.Username._value = $$._value</tag> </item> <item> <?MS_Grammar_Editor GroupWrap?> <ruleref uri="Library.grxml#Cancel" type="application/srgs+xml"/> <tag>$.Cancel = $.Cancel ? $.Cancel : {}; $.Cancel._value = $$._value</tag> </item> </one-of> </rule>

11. QA Controls • Interact with Prompts Database, TTS Engine, SR Engine • Prerecorded vs. TTS prompts • Prompts generated by combining recorded prompts or parts of recorded prompts • 4 different timeouts for recognition timing: Initial, Babble, End Silence, Max

12. DATABASE • A collection of interrelated data • DB Models • Entity - Relationship Model • Relational Model • Network Models • etc.

13. CIMS • A Relational DB using MS SQL Server 2000 • Entities →Relations→Tables • Users • Courses • etc.

14. SQL • String Query Language • Update, Insert, Delete etc. • Operations on CIMS • Make Announcements • Give Assignments • etc.

15. Voice-Only CIMS • Connect to CIMS Database • Query the necessary tables • Ex : SELECT ID, Code FROM COURSES WHERE Enabled='1‘ • Update the tables • Ex: UPDATE USERS SET LAST_ACCESS = GETDATE() WHERE USERNAME = ‘9902959'

16. Regular Expressions • Text Processing and String Manipulations • Pattern Matching • String Splitting • Substring Replacement. • Regex Class by C#.NET

17. Regex-CIMS • Free Labeling • Ex: hw#3, Homework 3, Spice hw 3 … • Using prerecorded prompts • homework 3 • Regex.Replace(name,@"(?<preamble>\w*\s*)(H|h)\w*(W|w)([^09]*(?<no>\d*)).*","${preamble}"+"homework "+"${no}")

18. Conlusion • An IVR system, operating side-by-side with online version of CIMS developed • Opportunity to gain knowledge about emerging technologies • Satisfaction of developing a working system with zero initial knowledge

19. What to do next? • Windows Server 2003 • Microsoft Speech Server (MSS) (released in May 2004) • Telephone card • Deployment onto MSS

20. Thanks for listening Murat Semerci Çağdaş Kayra Akman