Semantic Information Processing of Spoken Language - How May I Help You? sm

1. Semantic Information Processingof Spoken Language- How May I Help You? sm Alicia Abella AT&T Labs – Research Florham Park, New Jersey

2. Motivation • Goal is to provide automated customer services via natural spoken dialog. • Natural means what people actually say, rather than what we’d like them to. • Shift burden from user to machine

3. Why Spoken Language Understanding? • User interface as the bottleneck to exploiting speech and language processing technological advancement • Spoken language as focus of this work • Machine Initiative • Menus (please say collect, calling card …) • Classes (please say credit card number, destination city, person name, date, …) • User Initiative • How may I help you?

4. Stroustrup on programs which communicate with people "... it must cope with that person's whims, conventions and seemingly random errors. Trying to force the person to behave in a manner more suitable for the machine is often (rightly) considered offensive.” from "The C++ Programming Language”(1987) pp. 76

5. A History of Applications • Department Store Call-Routing (1989-1991) • Almanac Data Retrieval (1992) • Airline Travel (1993) • Multimodal Blocks World (1993) • Operator Services (1995-9) • Customer Care (2000+) • Enterprise Customers (2002+)

6. HowMay I Help You? SM • Prompt is “AT&T. How may I help you?” • User responds with unconstrained fluent speech • System recognizes and determines the meaning of users’ speech, then routes the call • Dialog technology enables task completion HMIHY . . . Local Account Balance Calling Plans Unrecognized Number

7. Extracting Meaning from Speech • Extracting meaning is primary in speech understanding systems. • How to quantify the information content of a natural language message? • Such theory is crucial to engineering devices which understand and act upon such messages.

8. Communication Paradigm • Goal of communication is to induce the machine to • perform some action • undergo some internal transformation • Communication is successful if the machine responds appropriately • Contrast with traditional communication theory

9. Shannon (1948) The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning, … These semantic aspects of communication are irrelevant to the engineering problem.

10. Architecture for Natural Spoken Dialog Voice reply to customer Speech Speech Text-to-SpeechSynthesis Automatic SpeechRecognition TTS ASR Data Words to be synthesized Words spoken Words Words SLG SLU Spoken Language Generation Spoken LanguageUnderstanding Meaning DM Action Meaning Dialogue Management

11. Architecture for Natural Spoken Dialog Play prompt ASR Dialog Manager Spoken Language Understanding User speech Language Models Acoustic Models Salient Grammar Fragments Inheritance Hierarchy

12. Technology Component Traits • Robustness • ASR • Large vocabulary > 10,000 words • Dialects (Nationwide deployment) • Real-time • SLU • Tolerance of varied phraseology • Many ways of saying the same thing • Similar way of saying different things • Real-time • Dialog • Confirmation, Re-prompting, Context switching • Say anything, anytime, anyway

13. Examples of Customer Utterances Account Balance Other General Billing Rates and Calling Plans Charge on Bill Change Customer Info

14. Example Dialogs • Rate Plan • Account Balance • Local Service • Unrecognized Number • Threshold Billing • Billing Credit

15. The technology is in use today • AT&T Customer Care organizations • Consumer service live since Nov. 2000 • Decreased servicing costs; reduced time spent in automation; reduced repeat calls and customer defections • Small Business service pilot underway • Supports 800#s used for billing inquiries and corporate calling card transactions. • Determines caller intent to perform activities such as making payments, requesting bill adjustments, ordering cards, reporting stolen cards • AT&T Enterprise customers • Several in beta trials • Delivers this functionality in a networked, managed environment.

16. Industry Specific Applications Check account balances, Apply for mortgage, Request credit report, Locate branch Verify coverage, Inquire about a claim, Check claim status Get store hours, Locate nearest store, Directions, Check inventory availability and order status Benefit enrollment, Get a referral, Obtain test results, Pre-admissions procedures Obtain a price quote, Make reservations, Get a seat assignment, Check flight status, Redeem miles Get instructions, Report a problem, Obtain problem status, Order And Applications Common to All Industries: Password Reset, PIN Reset, FAQs, Help Desk, Locator Services, Order Entry and Status

17. Key Value Determinants • Enhanced Customer • Experience • Reduced wait and call times • even at peak times • Natural efficient and • personalized dialog • Properly fulfills/routes • request the first time • Decrease Costs • Increased use of automation • reduces servicing costs • Liberate agent headcount • Reduce handling time and • hang-ups and call-backs • Enhanced Business • Results • Implement new applications • to drive revenue • No capital investment to • build or support • Dynamic, customizable • resource sharing in secure • environment

18. Observed Benefits • Increased automation • Improved routing • Ability to add more functionality • Improved customer satisfaction • Decreased repeat calls (37%) • Decreased customer defection rate (18%) • Decreased rep time per call (10%) • Decreased customer complaints (78%)

19. Research References at www.research.att.com/~algor/hmihye.g. IEEE Computer Magazine, April 2002