700 likes | 798 Views
Why should I care about Computational Linguistics & Language Processing?. Hsiao-Wuen Hon 洪小文 Assistant Managing Director Microsoft Research Asia. Should I care?. Medical school 金饭碗 Electronics 配股 Easy way to become millionaire Chip manufacture TSMC, UMC Hardware
E N D
Why should I care about Computational Linguistics & Language Processing? Hsiao-Wuen Hon 洪小文 Assistant Managing Director Microsoft Research Asia
Should I care? • Medical school • 金饭碗 • Electronics • 配股 • Easy way to become millionaire • Chip manufacture • TSMC, UMC • Hardware • Acer, Quanta, 鸿海, BenQ, 英业达, MiTac • NLP? Speech? IR? HWR?
It is actually a good choice • People go on to have good careers • Many applications • IR, HWR • Investment banks • Bioinformatics • ….. • With many smart people • Software Industry cares • Not overproducing students
Industry Cares • People you might know • Academics • Pillars of A.I. • Well funded • Taiwan professors • Oversea professors • V. Zue, B.H. Juang, F. Jelinek, M. Libermann, N. Chomsky, Michael Collin, Fernando Pereira …
Industry Cares • Industrial R&D Labs • Executives • Kai-Fu Lee (MS), Qi Lu (Yahoo), … • Microsoft • X.D. Huang,洪小文,马维英, Eric Chang, 周明, Eric Brill, Ken Church, … • Continue hiring • Google • Speech - Amit Singhal, Michael Riley, … etc., • NL – Franz Och, Krishna Bharat, Dekang Lin, … • Aggressively hiring • Others…
Industry Cares • Other applications • Renaissance Technologies • Hedge fund management – 4 billions in assets • Time-series predication based on S&L technologies • a.k.a ex-IBM S&L group • P. Brown, R. Mercer, P. De Souza, L. Bahl, Della Pietra brothers, … • Startups • Nuance, SpeechWorks, InfoTalk, iPhrase, Lexicus, …
Bill Gates’ vision PC on everyone’s desktop (’75) Information at your finger tips (’90) Seamless Computing (’03) S&L technologies is the key Billions of $ investment in S&L technologies Full-size S&L product & research groups Multi-lingual & multi-products Continue hiring Expanded investment due to search/Google Microsoft Cares
Information Agent • “Do what I mean” • “Find what I want” • How to turn on Firewall in Windows? • Speech recognition • Signal to text • Natural language understanding • Syntax/semantics • Domain knowledge • Knowledge search • AI-Complete
A Long Long Journey • Speech • Ubiquitous interface • Automatic Speech Recognition • Text-to-Speech • Natural Language • Spelling/grammar/style checking • IME • Machine translation • Information Retrieval & Mining
Speech • SAPI 1.0 – 6.0 • Window Sound System in ’92 • Platform for building speech app. in Windows • Accessibility support (Screen Reader) • Office Dictation • Chinese, English • Microsoft Speech Server • Telephony speech & multiomdal platform • Other – Encarta, WinCE/Smartphone…
30% Human Error Rate Machine Error Rate 25% Log (Machine Error Rate) 20% 15% 10% 5% 0% 1993 1996 1999 2002 2005 2008 2011 Speech
Mulan AT&T Loquendo Elan Speech MSRA Speech • TTS – multi-lingual natural TTS • ASR • Chinese LVCSR - dictation/telephony/embedded • Fundamental research • AIME: Audio Info. Management & Extraction • Audio/video file indexing/retrieval • Offline transcription/extraction/summarization • More in Eric’s keynote tomorrow • From the Lab to Ubiquity: Speech Technology's Road to Mainstream
NLP Contributes to MS Products • IME (Chinese, Japanese, …) • Spelling/grammar checking • Spam filtering • English Writing Wizard (EWW) • Spoken language interface • IR and CLIR • Text mining • Machine translation • Search engine • QA (AskMSR) • SLM for Speech • Text analysis for TTS • …..
NLP “Rainbow” Understanding Knowledge base Analysis Generation Discourse Discourse Logical Form Logical Form Transfer Syntax Syntax Dictionary Dictionary Morphology Morphology Word Breaking Grammar Checking Machine Translation Source Text Target Text
Translation evaluation paraphrasing Tran. know. acquisition Web retrieval Shallow MT Cross language IR Indexing EBMT & SMT MRD Balanced corpus QMapping MRD Bilingual corpus Parsing lexicon Tagged corpus Translation lexicon Bilingual tagged corpus Resume routing NLP at MSRA Applications Chinese IME English writing wizard Enterprise search Japanese IME Pocket translator SQL Text Mining Spelling check Extended TM Resume Routing NLP Machine Translation Information Extraction Information Retrieval Research Meta data extraction Skeleton parser Term extraction Named entity identification Annotation tool Pos tagging Machine learning SLM Monolingual resources (C, J, E) Bilingual resources (C, E) Special purpose Linguistic Resources
NLP at MSRA • TIME • Email Routing • Spam filtering • Resume routing • Support routing • EWW • Translation
TIME System TIME Platform • Text Information Management & Extraction • Goal: extract information from text data • genres: email, newspaper, report, web pages • formats: Word document, PDF/PS, HTML/XML • languages: English, Chinese, Japanese, … • Applications: search, question answering, data mining, machine translation
TIME Components • Linguistic processing TIME linguistic platform • Text normalization: sentence splitting, tokenization, morphological analysis • Entity extraction: person name, company name, time expression, phrases • Relation learning: syntactic/semantic dependencies between entities • Information extraction • Document property extraction: title, author, key term, summary • Domain knowledge extraction: concept, concept relation, glossary, taxonomy, event • Cross-lingual information exchange • Translation at word, entity, term, skeleton, text levels • Reading, writing, cross language information retrieval
Multi-lingual linguistic unit processing • Word • Tokenization • Named entity recognition (NER) • POS • Sentence • Chunking (VP/NP) • Source-channel models:
Skeleton Parser • Skeleton == <subject V object> • Input: He is succeeded by Ivan Allen Jr. • Output Obj Sub [He] is succeeded by [Ivan Allen Jr.] • More robust & faster than traditional parser • Adequate for most applications • Collocation checking, Spell checking, Grammar checking, QA, Search
Skeleton Parser • Key Dependency Relations • A set of most important relations (e.g. subject, object…) • Definition based on application • Our Target: A Robust&FastDependency Extractor • Not rely on high quality (hand-annotated) training data. • High efficiency in dealing with large scale of data (e.g. web data) • Potential Applications • Information Extraction, Q/A, TDT • Who (Subject-Verb), Whom (Verb-object), What (Adj-Noun) • Machine translation • Skeleton translation • NL-based Information Retrieval • Cross-Language IR • Re-ranking by triple matching
Proposed approach Input Sentence Raw corpus NLPWin Parser PoS Tagging Parsed corpus Chunking Noise Filtering Shallow Parser Training Data Key Dependency Triples Training
The proposed approach Input Sentence Raw corpus NLPWin Parser PoS Tagging Parsed corpus Chunking Noise Filtering Shallow Parser Training Data Key Dependency Triples Training
The proposed approach Input Sentence Raw corpus NLPWin Parser PoS Tagging Parsed corpus Chunking Noise Filtering Feature Extraction Training Data Classification Training Key Dependency Triples
Term Extraction Candidate Generation Ranking Term List Text Options: Term frequency TF-IDF Entropy reduction ER-IDF Options: Boundary determination BaseNP Pattern filtering Terms
Text Mining Roadmap Information Desk Meta Data for Sharepoint SQL Text Mining Text Miner • Key technologies • Metadata extraction • Ranking algorithm • Multi-languages support
Information Desk • http://msra-nlc-tm1
Machine Translation Roadmap • Direction • Template based • Linguistic data acquisition from Web mining • TIME Search Engine Office EWW Mobility • Key technologies • Skeleton parser • Collocation checker • Paraphrase • Knowledge acquisition • Adaptive to new language pairs
EWW (English Writing Wizard) Idiomatic Usage • Objectives • Make your English writing as good as native speakers Input: question question (Noun) Verb+question: raise ~, ask ~, resolve ~, pose ~ Adj+question: unanswered ~, serious ~, big ~, real ~ question (Verb) question+Noun: ~ motive, ~ value, ~ truth, ~ boy question+Adv: ~ intensely, ~ orally, ~ closely, ~ at_all Adv+question: privately ~, cautiously ~, hardly ~ • Features • Idiomatic usages • Synonymous collocation • Collocation translations • Bilingual example sentences Synonymous Collocation attain~dobj~level achieve~dobj~level attract~dobj~fan draw~dobj~fan take~dobj~reins assume~dobj~reins|hold~dobj~reins bad~Intnsifs~extremely risky~Intnsifs~extremely unusual~Intnsifs~quite unusual~Intnsifs~rather vision~Attrib~unusual sight~Attrib~unusual Improve~Mod~greatly Improve~Mod~considerably • Technology Highlights • Auto extraction of idiomatic usage • Auto extraction of synonymous collocation • Auto extraction of collocation translations • Example sentence retrieval Collocation Translation 克服~困难 conquer difficulty, overcome difficulty, master~difficulty overcome~adversity, surmount~difficulty
Web Search & Mining • Internet + Data + Information -> Search, Mining, Sharing, & Intelligence • Lots of text • Text-based IR • Text Mining • Semantic/Structure Mining • Media Search • Surrounding text • Audio/video transcription • Make Billions of $ from trillions of words
Information Retrieval • Text Processing • Tokenization • Normalization – stemming, … • Precision/Recall • Beyond 1st order statistics (TF-IDF) • N-gram for adaptive indexing • Better model of P(Doc|Query) • Classification vs. term frequency • Result Summarization • Query sensitive • U盘 (优盘) vs. 大拇哥 • Result clustering & classification
Search Long Result List • A user search for information about “jaguar”, a Mac OS • However, the relevant results are mixed with other pages • The user need to go through a long list to find desired information
Clustering vs. Classification Clustering Results for “jaguar” Classification Results for “jaguar”