Applications of Intelligent Systems and Robotics in Service of Society

1. Applications of Intelligent Systems and Robotics in Service of Society Raj Reddy Carnegie Mellon University Pittsburgh Jan 9, 2007 Keynote Speech at IJCAI 2007, Hyderabad, India

2. Outline of the Talk • Needs of Developing Economies • Access to Knowledge, Education and healthcare, etc. • 3 Minute Introduction to AI: What is it and how it can help • The role of AI in enabling • access to knowledge and knowhow • access to libraries • access to education and learning • access to health care • Unfinished research agenda of AI

3. Needs of the People with Per Capita Income of Less Than $1 a Day • Access to entertainment • watch any movie, TV show when desired • Telemedicine • providing links to doctors and treatment at a distance • Access to information • about hygiene and safe water, helping to reduce infant mortality • Life-long learning • independent of the limitations of language, distance, age and physical disabilities • Price discovery • Marketing assistance • using eBay like auction exchanges • Find jobs • e.g. monster.com They need AI and IT but not Word, Excel and Powerpoint

4. Barriers to Entry: The Digital Divide • Connectivity Divide • Access to free Internet for basic services? • Computer Access Divide • Accessibility: Less than 5 minute walk? • Affordability: Costing less than a cup of coffee per day? • Digital Literacy Divide • Language Divide • Literacy Divide • Content Divide • Access to information and knowledge • Access to health care • Access to education and learning • Access to jobs • Access to entertainment • Access to improved quality of life

5. A 3-Minute Introduction to AI • What is it and how it can help • review why the world’s poor have more to gain in relative terms by the effective use of the IT and AI technology

6. Artificial Intelligence attempts to make computers do things which would require intelligence in people, i.e. any activity which requires the use the human brain

7. A Historical View of Advances in AI • 1950s: Theorem Proving; Chess • 1960s: Problem Solving; Language: Understand; Question Answering • 1970s: Speech; Vision; Expert Systems • 1980s: Robotics; Knowledge Based Systems • 1990s: Language Translation; Search • 2000s: Systems that Learn with Experience

8. Some Application Domains • Web Search : Google, Yahoo, MSN • Intelligent car • Financial planning • Manufacturing control • System diagnosis • NL communicator • Writing assistant • Knowledge-based simulation • Games • Household robot

9. Requirements for Intelligence • Learn from experience • Exploit vast amounts of knowledge • Exhibit Goal Directed Behavior • Tolerate error and ambiguity in input • Communicate with natural language • Operate in real time, and • Use symbols (and abstractions)

10. AI Problem Domains & Attributes Puzzles Chess Theorem Proving Expert Systems Natural Language Motor Processes Speech Vision Knowledge Data Response Content RateTime Poor Low Hours Rich High Real Time

11. Lessons from AI Experiments • Bounded Rationality implies Opportunistic Search • An Expert becomes a World Class Expert only after spending at least 15 years of intensive practice and knows 70,000+20,000 patterns • Search Compensates for Lack of Knowledge • Knowledge Compensates for Lack of Search • A Physical Symbol System is Necessary and Sufficient for Intelligent Action

12. How Can AI Help? • Intelligent Systems in support of • Access to Knowledge and Knowhow • Learning and Education • Health • Robotics for Accident Avoiding Cars, Landmine Detection, and Disaster Recovery

13. Enabling Access to Knowledge and Information

14. Village Google: Access to Knowledge for Use in a Village • Access to Essential Information and Advice • Medical, Agriculture, FAQ indexed and searchable • Interactive access to Doctors, Rescue Personnel • Lifelong Learning and Education • Agricultural Information • Price discovery, crop disease information, weather prediction • Access to Markets and Jobs • Disaster Relief and Management • Access to Newspapers, Radio and TV • Entertainment and Amusement • Communications • Video Phone, IP Telephone, Instant Messaging • Video Email, Voice Email, Text Email

15. The Vision of a Global Knowledge Network • Create a Knowledge Networkthat connects experts to the people who need help, e.g., farmers in villages • End-users interact at Village Knowledge Centers • Equipped with a networked computer and basic A/V equipment • Staffed by a Knowledge Officer • Humans are intrinsic to Knowledge Networks (raw information  knowledge!) • Domain experts provide answers to previously unanswered questions • Answers converted into an “encyclopedia-on-demand” video documentary at higher-level centers centers and dubbed into local languages in each country • Also available for direct access browsing by literate and networked users

16. System Overview Multi-level Information Flow - An example scenario • Hierarchical structure spanning districts, regions, countries, etc. • Outside experts interact with higher level Knowledge Officers • Builds up an ever-increasing multimedia database • Can provide static (e.g., best-practices) as well as dynamic (e.g., weather, prices, etc.) information • Innovative mechanisms and processes for information digitization, exchange, analysis, and dissemination An illiterate farmer goes to a Village Knowledge Officer (with a computer connected to FAO multimedia database) and asks a question in his or her local language The KO retrieves answer from local Multilingual database within minutes 80 - 90% of the time For the remaining 10 - 20% of the time the KO puts up the question to a higher level office and gets an answer back, typically in less than 24 hrs 100s of domain experts populate the databases, both as part of their jobs and as volunteers (say, 2 questions per week)

17. Knowledge officers and Domain Experts World Knowledge Management & Coordination (global) Nation Knowledge Management & Coordination (national level) Expertise of Knowledge Officers State Verification of Query-Answer Relevance And RFP to domain experts District Translation, Information Retrieval Village AV data collection, Transliteration and Transcription Information Retrieval Domain experts: Volunteer to answer at least 2 questions a week (or part of job responsibility)

18. Roles of Knowledge Officers Village District (sub)continent Global Region/Nation 3,000 people Transcription (and possibly Transliteration) 300,000 people Translation and Information Retrieval 30M people Verification & RFP from Experts 0.3B people Knowledge Management & Coordination 3 Billion people Knowledge Analysis and Inference Records question of the end-user in audio-video format. Enters text transcription of the question. Searches local language database for answer Need not be knowledgeable in English. Enters translation of questions. Searches multilingual database for answer Sends answer after translation to lower level If question not among FAQs or automated system, sends to higher level Picks questions of critical nature and validates the answer provided at lower level If critical or unanswered question, puts up request to experts even if not paid for by end-user Same as next level up, but with the range of analyses broadened to the region/subcontinent level Brings experts to where their knowledge is needed. Mobilization of resources towards their need. Identifies and triggers initiatives to control “epidemic”-like problems (All numbers shown are for rural, developing country populations = beneficiaries)

19. The AI Challenges in Creating a Global Knowledge Network • Farmers typically not able to tap in to existing networks • Often illiterate • Rarely have relevant information or even communications accessible • Today’s Internet and existing databases/portals are primarily intended for users literate in English and can synthesize their solutions from multiple sources

20. Internet Bill of RightsJaime Carbonell, 1994 • Get the right information • e.g. search engines • To the right people • e.g. categorizing, routing • At the right time • e.g. Just-in-Time (task modeling, planning) • In the right language • e.g. machine translation • With the right level of detail • e.g. summarization • In the right medium • e.g. access to information in non-textual media

21. Relevant Technologies search engines classification, routing anticipatory analysis machine translation summarization speech input and output • “…right information” • “…right people” • “…right time” • “…right language” • “…right level of detail” • “…right medium”

22. “…right information”Search Engines

23. The Right Information • Right Information from future Search Engines • How to go beyond just “relevance to query” (all) and “popularity” • Eliminate massive redundancy e.g. “web-based email” • Should not result in • multiple links to different yahoo sites promoting their email, or even non-Yahoo sites discussing just Yahoo-email. • Should result in • a link to Yahoo email, one to MSN email, one to Gmail, one that compares them, etc. • First show trusted info sources and user-community-vetted sources • At least for important info (medical, financial, educational, …), I want to trust what I read, e.g., • For new medical treatments • First info from hospitals, medical schools, the AMA, medical publications, etc. , and • NOT from Joe Shmo’s quack practice page or from the National Enquirer. • Maximum Marginal Relevance • Novelty Detection • Named Entity Extraction

24. Beyond Pure Relevance in IR • Current Information Retrieval Technology Only Maximizes Relevance to Query • What about information novelty, timeliness, appropriateness, validity, comprehensibility, density, medium,...?? • Novelty is approximated by non-redundancy! • we really want to maximize: relevance to the query, given the user profile and interaction history, • P(U(f i , ..., f n ) | Q & {C} & U & H) where Q = query, {C} = collection set, U = user profile, H = interaction history • ...but we don’t yet know how. Darn.

25. Maximal Marginal Relevance vs. Standard Information Retrieval documents query MMR Standard IR IR

26. “…right people”Text Categorization

27. The Right People • User-focused search is key • If a 7-year old is working on a school project • taking good care of one’s heart and types in “heart care”, she will want links to pages like • “You and your friendly heart”, • “Tips for taking good care of your heart”, • “Intro to how the heart works” etc. • NOT the latest New England Journal of Medicine article on “Cardiological implications of immuo-active proteases”. • If a cardiologist issues the query, exactly the opposite is desired • Search engines must know their users better, and the user tasks • Social affiliation groups for search and for automatically categorizing, prioritizing and routing incoming info or search results. New machine learning technology allows for scalable high-accuracy hierarchical categorization. • Family group • Organization group • Country group • Disaster affected group • Stockholder group

28. Text Categorization Assign labels to each document or web-page • Labels may be topics such as Yahoo-categories • finance, sports, NewsWorldAsiaBusiness • Labels may be genres • editorials, movie-reviews, news • Labels may be routing codes • send to marketing, send to customer service

29. Text Categorization Methods • Manual assignment • as in Yahoo • Hand-coded rules • as in Reuters • Machine Learning (dominant paradigm) • Words in text become predictors • Category labels become “to be predicted” • Predictor-feature reduction (SVD, 2, …) • Apply any inductive method: kNN, NB, DT,…

30. “…right timeframe”Just-in-Time - no sooner or later

31. Just in Time Information • Get the information to user exactly when it is needed • Immediately when the information is requested • Prepositioned if it requires time to fetch & download (eg HDTV video) • requires anticipatory analysis and pre-fetching • How about “push technology” for, e.g. stock alerts, reminders, breaking news? • Depends on user activity: • Sleeping or Don’t Disturb or in Meeting  wait your chance • Reading email  now if info is urgent, later otherwise • Group info before delivering (e.g. show 3 stock alerts together) • Info directly relevant to user’s current task  immediately

32. “…right language”Translation

33. Access to Multilingual Information • Language Identification (from text, speech, handwriting) • Trans-lingual retrieval (query in 1 language, results in multiple languages) • Requires more than query-word out-of-context translation (see Carbonell et al 1997 IJCAI paper) to do it well • Full translation (e.g. of web page, of search results snippets, …) • General reading quality (as targeted now) • Focused on getting entities right (who, what, where, when mentioned) • Partial on-demand translation • Reading assistant: translation in context while reading an original document, by highlighting unfamiliar words, phrases, passages. • On-demand Text to Speech • Transliteration

34. Knowledge-Engineered MT Transfer rule MT (commercial systems) High-Accuracy Interlingual MT (domain focused) Parallel Corpus-Trainable MT Statistical MT (noisy channel, exponential models) Example-Based MT (generalized G-EBMT) Transfer-rule learning MT (corpus & informants) Multi-Engine MT Omnivorous approach: combines the above to maximize coverage & minimize errors “…in the Right Language”

35. “…right level of detail”Summarization

36. Right Level of Detail • Automate summarization with hyperlink one-click drilldown on user selected section(s). • Purpose Driven: summaries are in service of an information need, not one-size fits all (as in Shaom’s outline and the DUC NIST evaluations) • EXAMPLE: A summary of a 650-page clinical study can focus on • effectiveness of the new drug for target disease • methodology of the study (control group, statistical rigor,…) • deleterious side effects if any • target population of study (e.g. acne-suffering teens, not eczema suffering adults ….depending on the user’s task or information query

37. Information Structuring and Summarization • Hierarchical multi-level pre-computed summary structure, or on-the-fly drilldown expansion of info. • Headline <20 words • Abstract 1% or 1 page • Summary 5-10% or 10 pages • Document 100% • Scope of Summary • Single big document (e.g. big clinical study) • Tight cluster of search results (e.g. vivisimo) • Related set of clusters (e.g. conflicting opinions on how to cope with Iran’s nuclear capabilities) • Focused area of knowledge (e.g. What’s known about Pluto? Lycos has good project in this via Hotbot) • Specific kinds of commonly asked information(e.g. synthesize a bio on person X from any web-accessible info)

38. Document Summarization Types of Summaries

39. “…right medium”Finding information in Non-textual Media

40. Indexing and Searching Non-textual (Analog) Content • Speech  text (speech recognition) • Text  speech • TTS: FESTVOX by far most popular high-quality system • Handwriting  text (handwriting recognition) • Printed text  electronic text (OCR) • Picture  caption key words (automatically) for indexing and searching • Diagram, tables, graphs, maps  caption key words (automatically)

41. AI and Access to LibrariesThe Million Book Digital Library Project

43. One Step at a Time… • Million Book DL • Only about 1% of all the world’s books • Harvard University 12M • Library of Congress 30M • OCLC catalog 42M • All Multilingual Books ~100M • At the rate of digitization of the last decade it would take a 100 years!

44. Million Book Project: Issues • Time • At one page per second (20,000 pages per day shift), it will take 100 years (200 working days per year) to scan a million books of 400 pages each • Cost • 100M books at US$100 per book would coat $10B • Even in India and China the cost will be $1B • The annual cost is currently expected to be close $10M per year with support from US, India and China. • Selection • Selection of appropriate books for scanning is time consuming and expensive

45. Million Book Project: Issues (cont) • Logistics • Each containers hold 10,000 to 20,000 books. Shipping and handling costs about $10,000 • Meta Data • Accessing and/or creating Meta data requires professionals trained in Library science • Optical Character Recognition Technology • Essential for searching, translation and summarization • Many languages don’t have OCR

46. Million Book Project: Status • 18 Centers in India • 22 centers in China • 1 Center in Egypt • 15 Centers in Poland • Planned : Australia • Over 1,400,000 books scanned • Over 250,000+ accessible on the web