1 / 40

Watson

Watson. Robert Yates Watson Core Development. Agenda. A brief History of Watson What is it good for? How does it work? Current Focus. A Brief History of Watson. A Brief History of Watson. Deep Blue Ended in 1997 Looking for a new research challenge

montana
Download Presentation

Watson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Watson Robert Yates Watson Core Development

  2. Agenda • A brief History of Watson • What is it good for? • How does it work? • Current Focus

  3. A Brief History of Watson

  4. A Brief History of Watson • Deep Blue Ended in 1997 • Looking for a new research challenge • 2004, IBM Research manager Charles Lickel, over dinner with coworkers, noticed that the restaurant they were in had fallen silent • Ken Jennings • Started in Research in 2005 • Won Jeopardy In Feb 2011 • SW Division in Aug 2011

  5. What is it good for?

  6. Watson’s real value proposition: Efficient decision support over unstructured (and structured) content Deeper Understanding,Higher Precision and Broader,Timely Coverage at lower costs Open-Domain Question-Answering Jeopardy! Challenge Existing BI Relevance Ranking Inference/Rules Key WordSearch SQL/XQuery Shallow Understanding Low Precision Broad Coverage Deeper Understanding but Brittle High Precision at High Cost Narrow Limited Coverage Unstructured Data • Broad, rich in context • Rapidly growing, current • Invaluable yet under utilized Structured Data • Precise, explicit • Narrow, expensive 7

  7. How does it work?

  8. Learning To Rank Systems • Training data is used by a learning algorithm to produce a ranking model which computes relevance of responses for actual queries. • Typically, users expect a search query to complete in a short time, which makes it impossible to evaluate a complex ranking model on each document in the corpus, and so a two-phase scheme is used. First, a small number of potentially relevant documents are identified using simpler retrieval models which permit fast query evaluation. In the second phase, a more accurate but computationally expensive machine-learned model is used to re-rank these documents.

  9. Learning To Rank Systems • Training data is used by a learning algorithm to produce a ranking model which computes relevance of responses for actual queries. • Typically, users expect a search query to complete in a short time, which makes it impossible to evaluate a complex ranking model on each document in the corpus, and so a two-phase scheme is used. First, a small number of potentially relevant documents are identified using simpler retrieval models which permit fast query evaluation. In the second phase, a more accurate but computationally expensive machine-learned model is used to re-rank these documents.

  10. Trained Models Hypothesis Evidence Scoring Hypothesis Evidence Scoring Hypothesis Evidence Scoring Final Merging Ranking Scoring Search Results Learning To Rank (Basic Architecture) Keywords

  11. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Learning To Rank (Watson) Wikipedia etc. Evidence Retrieval

  12. Who is the 44th President of the United States?

  13. Question Analysis Who is the 44th President of the United States? Who is the 44th President of the United States?

  14. Question Analysis Who is the 44th President of the United States? 'Who' is the '44th' 'President' of the 'United States'? Who is the 44th President of the United States? Lexical Answer Type → Person Keywords Focus Can be replaced by the correct answer to make a true statement

  15. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc. Evidence Retrieval

  16. Question Question Analysis Search Who is the 44th President of the United States? Keywords: 44th President United States Primary Search

  17. Question Question Analysis Search Who is the 44th President of the United States? Keywords: 44th President United States Primary Search

  18. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc. Evidence Retrieval

  19. Candidate Answer Generation Question Question Analysis Search Who is the 44th President of the United States? Primary Search

  20. Candidate Answer Generation Question Question Analysis Search Who is the 44th President of the United States? Barack Obama George W. Bush Harvard Law School Illinois Primary Search

  21. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc. Evidence Retrieval

  22. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Who is the 44th President of the United States? Barack Obama Who is the 44th President of the United States? George W. Bush Who is the 44th President of the United States? Harvard Law School Who is the 44th President of the United States? Illinois Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Answer Scoring Answer Scoring

  23. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Who is the 44th President of the United States? Barack Obama Who is the 44th President of the United States? George W. Bush Who is the 44th President of the United States? Harvard Law School Who is the 44th President of the United States? Illinois Who is the 44th President of the United States? → Person Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Answer Scoring Answer Scoring

  24. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Who is the 44th President of the United States? Barack Obama Who is the 44th President of the United States? George W. Bush Who is the 44th President of the United States? Harvard Law School Who is the 44th President of the United States? Illinois Who is the 44th President of the United States? → Person Is Barack Obama a Person? .90 Is George W. Bush a Person? .90 Is Harvard Law School a Person? .10 Is Illinois a Person? .15 Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Answer Scoring Answer Scoring

  25. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc. Evidence Retrieval

  26. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Contextual Answer Scoring Contextual Answer Scoring Contextual Answer Scoring

  27. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Contextual Answer Scoring Barack Obama is the 44th President of the United States George W. Bush is the 44th President of the United States Harvard Law School is the 44th President of the United States Illinois is the 44th President of the United States Contextual Answer Scoring Contextual Answer Scoring

  28. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Contextual Answer Scoring Barack Obama is the 44th President of the United States George W. Bush is the 44th President of the United States Harvard Law School is the 44th President of the United States Illinois is the 44th President of the United States Contextual Answer Scoring Contextual Answer Scoring

  29. Question Scoring Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Barack Hussein Obama II (i/bəˈrɑːkhuːˈseɪnoʊˈbɑːmə/; born August 4, 1961) is the 44th and currentPresident of the United States. George Walker Bush (born July 6, 1946) is an American politician who served as the 43rd President of the United States from 2001 to 2009 and the 46th Governor of Texas from 1995 to 2000. Contextual Answer Scoring Barack Obama .95 George W. Bush .80 Harvard Law School .05 Illinois.10 Barack Obama is the 44th President of the United States George W. Bush is the 44th President of the United States Harvard Law School is the 44th President of the United States Illinois is the 44th President of the United States Contextual Answer Scoring Contextual Answer Scoring

  30. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc. Evidence Retrieval

  31. Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Evidence Retrieval

  32. Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Who is the 44th President of the United States? Evidence Retrieval

  33. Contextual Answer Scoring Candidate Answer Generation Answer Scoring Primary Search Contextual Answer Scoring Answer Scoring Contextual Answer Scoring Answer Scoring Trained Models Question Scoring Final Merging Ranking Scoring Question Analysis Search Scoring Answer, Confidence, Evidence Who is the 44th President of the United States? Wikipedia etc.

  34. IEEE Xplore Library: “This is Watson” • http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717 • Recommended • “This is Watson”, For all: at least read this thoroughly. • “Question Analysis, How Watson Reads a Clue” • “Finding needles in the haystack, Search and Candidate Generation” • “Automatic Knowledge Extraction from Documents” • “Structured Data and Inference in DeepQA” • “A framework for merging and ranking of answers in DeepQA” • Read the others to broaden your knowledge

  35. Current Focus

  36. Current Customer Engagements • Medicine • Challenge: Body of Knowledge doubles every 5 years • Best Practice / Evidence Based Medicine • Oncology Treatments • Finance

  37. Evidence Based Medicine • http://www.anthem.com/medicalpolicies/policies/mp_pw_a053323.htm • Varicose Veins • Medically Necessary: • Endoluminal radiofrequency ablation or endoluminal laser ablation, of the greater saphenous vein (GSV) or lesser saphenous veins (LSV) is medically necessary when the following criteria are met: • Saphenofemoral or saphenopopliteal junction incompetence (i.e., reflux) is demonstrated by Doppler or duplex ultrasound scanning; and • One or more of the following criteria (a, b, or c) are met: • Symptoms of venous insufficiency or recurrent thrombophlebitis (including but not limited to: aching, burning, itching, cramping, or swelling during activity or after prolonged sitting) which: • are interfering with activities of daily living; and • persist despite appropriate non-surgical management; and • persist despite a trial of properly fitted gradient compression stockings for at least six weeks;
or • There is ulceration secondary to stasis dermatitis; or • There is hemorrhage from a superficial varicosity.

  38. Treatment Example (Not Oncology) • A 44-year-old businessman presents to a physician because of a markedly inflamed and painful right great toe. He states that he just returned from a convention, and noticed increasing pain in his right foot during his plane trip home. Physical examination is remarkable for swelling and erythema of the right great toe as well as small nodules on the patient's external ear. Aspiration of the metatarsal-phalangeal joint of the affected toe demonstrates needle-shaped negatively birefringent crystals. • What’s wrong with this patient? • How do you treat it?

  39. Questions?

More Related