1 / 16

עיבוד שפות טבעיות - שיעור שישי Language Modeling

עיבוד שפות טבעיות - שיעור שישי Language Modeling. אורן גליקמן המחלקה למדעי המחשב אוניברסיטת בר אילן. מה המילה הבאה?. מנורת החסרונות (ר' נחמן מברסלב). ואת. אביו. בית. את. שעזב. אחד. באדם. מעשה. שם. ללמוד. רחוקות. למדינות. ויצא. מולדתו. ארץ. ולבית. לארצו. חזר. שנים. לאחר.

loyal
Download Presentation

עיבוד שפות טבעיות - שיעור שישי Language Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. עיבוד שפות טבעיות - שיעור שישיLanguage Modeling אורן גליקמן המחלקה למדעי המחשב אוניברסיטת בר אילן 88-680

  2. מה המילה הבאה? • מנורת החסרונות (ר' נחמן מברסלב) ואת אביו בית את שעזב אחד באדם מעשה שם ללמוד רחוקות למדינות ויצא מולדתו ארץ ולבית לארצו חזר שנים לאחר ומלאכות. חכמות אביו. 88-680

  3. למה זה טוב? • זיהוי קול • זיהוי כתב יד • זיהוי טעויות כתיב • סטיבן הוקינג 88-680

  4. למה זה טוב... • The camel lives in the dessert. • What is the whether forecast for today? • Give to books two that guy. • I ate a beach yesterday. 88-680

  5. ההסתברות של רצף מילים • P(w1,w2,…,wn) = P(w1)P(w2|w1)P(w3|w1w2w3)P(wn|w1… wn-1) 88-680

  6. The Noisy Channel Model 88-680

  7. Language Model • In the statistical language applications knowledge of the source if referred to as a language model. • A Language Model is a probability distribution over all possible utterances in the language. 88-680

  8. Back to Markov • In a trigram setting: • P(wn|w1… wn-1) = P(wn|wn-1 wn-2) • P(w1,w2,…,wn) =  P(wi| wi-1 wi-2) • Use MLE estimates from a corpus. 88-680

  9. The magic number - 3 • Why 3? • ככל שמסתכלים על מספר מילים גדול יותר לפני המילה הנוכחית, ההערכה מדויקת יותר. • ככל שמסתכלים על יותר מילים, קשה יותר לקבל נתונים סטטיסטיים מדויקים. 88-680

  10. Unigram approximation to Shakespeare • To him swallowed confess hear both. Which. Of save on trail for are ay device and rote life have • Every enternow severally so, let • Hill he late speaks; or! a more to leg less first you enter • Will rash been and by I the me loves gentle me not slavish page, the and hour; ill let • Are where exeunt and sighs have rise excellency took of.. Sleep knave we. near; vile like 88-680

  11. Bigram approximation to Shakespeare • What means, sir. I confess she? then all sorts, he is trim, captain. • Why dost stand forth thy canopy, forsooth; he is this palpable hit the King Henry. Live king. Follow. • What we, hath got so she that I rest and sent to scold and nature bankrupt, nor the first gentleman? • Enter Menenius, if it so many good direction found'st thou art a strong upon command of fear not a liberal l argess given away, Falstaff! Exeunt • Thou whoreson chops. Consumption catch your dearest friend, well, and I know where many mouths upon my undoi ng all but be, how soon, then; we'll execute upon my love's bonds and we do you will? • The world shall- my lord! 88-680

  12. Trigram approximation to Shakespeare • Sweet prince, Falstaff shall die. Harry of Monmouth's grave. • This shall forbid it should be branded, if renown made it empty. • What is't that cried? • Indeed the duke; and had a very good friend. • Fly, and will rid me these news of price. Therefore the sadness of parting, as they say, 'tis done. • The sweet! How many then shall posthumus end his miseries. 88-680

  13. Quadrigram approximation to Shakespeare • King Henry. What! I will go seek the traitor Gloucester. Exeunt some of the watch. A great banquet serv'd in ; • Will you not tell me who I am? • It cannot be but so. • Indeed the short and the long. Marry, 'tis a noble Lepidus. • They say all lovers swear more performance than they are wont to keep obliged faith unforfeited! • Enter Leonato's brother Antonio, and the rest, but seek the weary beds of people sick. 88-680

  14. From the Wall Street Journal • UnigramMonths the my and issue of year foreign new exchange's september were recession exchange new endorsed a acquire to six executives • BigramLast December through the way to preserve the Hudson corporation N. B. E. C. Taylor would seem to complete the maj or central planners one point five percent of U. S. E. has already old M. X. corporation of living on information such as more frequently fishing to keep her • TrigramThey also point to ninety nine point six billion dollars from two hundred four oh six three percent of the rates o f interest stores as Mexico and Brazil on market conditions 88-680

  15. Smoothing • Many perfectly reasonable trigrams will not appear in the corpus and will be assigned a zero probability. 88-680

  16. P(w) Max. Likelihood Estimate Smoothed LM (linear interpolation) w Language Model Smoothing (Illustration) 88-680

More Related