1 / 10

David Claiborn

SLM Tuning: Lessons Learned. David Claiborn. Agenda:. Define Statistical Language Model (SLM) Advantages and disadvantages of SLM technology How an SLM is used at Sprint/Nextel Practical SLM Tuning considerations Is an SLM right for your speech project? Questions.

dayo
Download Presentation

David Claiborn

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SLM Tuning: Lessons Learned David Claiborn

  2. Agenda: • Define Statistical Language Model (SLM) • Advantages and disadvantages of SLM technology • How an SLM is used at Sprint/Nextel • Practical SLM Tuning considerations • Is an SLM right for your speech project? • Questions

  3. What is the Statistical Language Model? Definition of an SLM: A statistical language model is a probabilistic description of the constraints on word order found in a given language .(Bahl et al 1983) • For our purposes think of an SLM as the probability of utterances occurring in a particular dialog state. This probability is created from caller utterances captured in that specific dialog state. • Traditionally, SLM technology is employed to give callers the ability to make requests using natural or conversational speech. SLM dialog states are often referred to as “Say Anything” states.

  4. Advantages and Disadvantages of an SLM over a finite state grammar Advantages: • Flexibility to callers • Able to serve natural speech requests • Minimized need for guidance from prompting Disadvantages: • Difficult to train and update • Transcription must be even more precise • Cost • Time

  5. SLM Application at Sprint/Nextel “Phone” • At Sprint/Nextel the SLM is literally the front door into the IVR. • In the diagram below we can see the SLM offers unique treatment to seven different “phone” centered requests. “My phone won’t make calls.” “I have a question about my phone.” “I want to buy a new phone.” “My phone is broken.” “I want to buy a phone charger.” “I lost my phone.”

  6. Things to consider when Tuning an SLM: • Does the SLM need a new destination or training to fulfill design requirements? • When training the SLM, what is a statistically relevant number of utterances to train on? • Do I have the expertise to tune this Say Anything state in house? • Do I have quality transcription in place? Have they guaranteed to maintain a certain level of accuracy (above 98%)? • Have I established a baseline to judge post tuning improvement?

  7. Is an SLM right for your speech project? At these were the initial questions IBM Global Services asked Sprint which led to the creation of Sprint’s SLM: • How many applications does the Customer Care IVR have today and what additional apps do you hope to add in the next five years. • How many callers enter the Customer Care IVR in a give year, what are the high and low months and are there certain months or times of each month where certain requests increase? • What level of call routing granularity are you looking to accomplish? • How rapidly will this system need to be taking calls? • What are your goals; increased CSAT and Call Completions, decreased agent to agent transfers?

  8. Questions?

  9. Bibliography: Bahl, L.R., Jelinek, F. & Mercer, R.L. (1983) "A Maximum Likelihood Approach to Continuous Speech Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, 5 (2), pp 179-190

  10. David Claiborn VUI Designer and Tuner claiborn@us.ibm.com

More Related