Back to The Future Divining the Future of AI

1. Back to The FutureDivining the Future of AI Raj Reddy Carnegie Mellon University May 31, 2018 Talk at MSRA, Beijing

2. AI Revisited:Misconceptions and Hype

3. What is AI? • AI Does Not Attempt to Replace Humans • Misconception leading to Many Outrageous Statements • AI Will Kill Us All • AI Will Enslave Us • AGI and ASI: Stupid Terminology • Singularity is a Myth • The Role of Engineering is the Enhance the Physical Capabilities of Humans • The Role of AI is to Enhance the Mental Capabilities of Humans • Human Machine Symbiosis • AI Can Solve Some Problems That Humans Cannot • Big Data Driven Discoveries • Humans Can Solve Some Problems That Machines Cannot • Together They Can Solve Problems Faster Better and/or Cheaper

4. What is AI? • AI is an attempt to automate tasks that are usually though to be uniquely Human • Requiring Intelligence, Intuition, Creativity, Innovation, Emotion, Empathy • Usually Human coding using Heuristics, Rules, and • Statistical Models - HMMs • Non Sequential Algorithms • Early Attempts • Attempts to Solve Problem not Expressible as Classical Algorithms • Proving Theorems, Playing Chess • Understand Language, Speech, Images • Create Robots that Sense Think and Act • Later Systems attempted • Compose music, Painting • Automate trading in Stock Market • Usually leads to Imperfect Solutions • OCR error rates 99% • 2010 Flash Crash: Dow Lost 1000 points in 36 minutes

5. AI In 20th Century: Use of Knowledge in Problem Solving • An Intelligent System must • Learns from Experience • Use Vast Amount of Knowledge • Tolerate Error and Ambiguity • Respond in Real Time • Communicate with Humans using Natural Language • Search Compensates for Lack of Knowledge • Puzzles • Knowledge Compensates for Lack of Search • F=ma; E = mc2 • Traditional Sources of Knowledge • Formal Knowledge: Learnt in Schools and Universities • Books, Manuals, • Informal Knowledge: Heuristics from Peoples and Environment • Human Encoding of Knowledge • Expert Systems • Knowledge Based System • Rule Based Ssytem

6. AI in 21st Century: Discover and Use Data Driven Knowledge Sources • Paradigm Shift in Science • First 3 Paradigms: Experiment, Theory, Simulation • Rutherford, Bohr, Oppenheimer • 4th Paradigm: Data Driven Science • Create Next Generation AI systems • Data Driven AI systems • To Solve Previously Unsolved Problems • Previously Unavailable Sources of Data • Knowledge from Big Data: • Data Driven Learning of Models and Algorithms • Knowledge from Multiple (Cross) Media: • Social Media Intelligence Gathering • From All Language Sources • From All the Media: Text, Speech, Image and Video • Knowledge from Crowd Intelligence: • Global Brain: from Individual Intelligence to Collective Intelligence • Knowledge from Augmented Intelligence: • Human-Machine Hybrid Intelligence for Collaborative Problem Solving • Knowledge from Unmanned Autonomous Vehicles: • Intelligence from Collaborating Teams of Robots • Automatic Discovery of New Knowledge • Machine Learning using Big Data • Deep Learning

7. Early Years of AI

8. 1960s: The Golden Age of Stanford AI Lab • Robotics • Computer Vision • Knowledge Engineering • Speech • Language Understanding • Computer Music • Chess, Symbolic Mathematics, Correctness of Programs, Theorem Proving, Logical AI, Common Sense • Time Sharing • LISP • DEC Clones: Foonly, Graphical Editors, Pieces of Glass, Theory of Computation

9. The Hand Eye Project • Interaction with the Physical World • Early work by • Karl Pingle, Bill Wichman, Don Pieper • Main Project Team • Jerry Feldman, R. Lou Paul, Marty Tenenbaum, Gerry Agin, Irwin Sobel, etc. • Robotic Hands • Bernie Roth and Vic Scheinman • Started in 1965 • Using the PDP1 and later the PDP6 • Led to Machine Vision and Robotics Industry • Via SRI and Vic Scheinman

10. Image Analysis and Understanding • Image Analysis • Manfred Hueckel, Ruzena Bajcsy, and Tom Binford • Led to Vision and Robotics at UPenn • Image Understanding • Natural Scenes and Face Recognition • Mike Kelly and Raj Reddy • Led to Vision and Robotics at CMU

11. Mobile Robotics • Mars Rover and Stanford Cart • Marvin Minsky (visiting) • Mars Explorer project 1964 • Les Earnest • Bruce Baumgart • Lynn Quam • Hans Moravec • Rod Brooks (later in the seventies) • Influenced direction of programs at CMU, MIT and SRI

12. Capturing Expertise • Heuristic Dendral: Representation, acquisition and use of knowledge in chemical inference • Project Team • Ed Feigenbaum, Josh Lederberg, Bruce Buchanan, Georgia Sutherland et al. • Started in 1965 • Led to • Expert Systems, Knowledge Engineering • Knowledge Based Systems Industry • Early Applications of AI

13. Speech • Speech Input to Computers • Started in 1964 as a class project • Using a PDP1 with drum memory and a display • By the end of 1964 we had a vowel recognizer running • Project team in the sixties • Raj Reddy, Pierre Vicens, Lee Erman, Gary Goodman, Richard Neely • Led to the DARPA Speech Understanding Project during the years 1971-76 • Led to CMU Advances in Speech and MSRA • Kai-Fu Lee • Hsiao-Wuen Hon • Xuedong Huang • Mei-Yuh Hwang • Most influential branch of Speech Recognition Industry: Dragon Systems, Apple, Microsoft • Indirectly IBM and Bell Labs

14. Language Understanding • Parsing and Understanding of Natural Language: Question Asking and Dialog Modeling • Computer Simulation of Belief systems • Ken Colby, Lawrence Tesler, Horace Enea et al • Parsing of Non-Grammatical Sentences • Colby, Enea et al • Conceptual Parsing • Roger Shank • Led to Language Processing Industry • via Shank and associates • Led to other Language Processing groups at Yale and UCLA • CMU, UMass, Berkeley, etc. • Influential strand of Language research

15. Computer Music • Computer Synthesis of Music • Started in 1964 on PDP1 • John Chowning • Leland Smith • Andy Moorer • Impact • Led to Yamaha adopting digital synthesis for consumer products • Establishment of a Center in Computer Music in Paris

16. Other AI Projects • Chess and other game playing programs • Kalah: R. Russell • Chess: McCarthy, Barbara Huberman (Liskov) • Checkers: Art Samuels • Symbolic Mathematics • Algebraic Simplification: Wooldridge and Enea • Reduce: Tony Hearn • Proving Correctness of Programs • Correctness of Programs: McCarthy and Painter • Equivalence of Programs: Kaplan and Ito • Properties of Programs: Zohar Manna • Theorem Proving • David Luckham and John Allen • Use of Predicate Calculus as a Representation for AI • McCarthy, Cordell Green et al • AI and Philosophy • McCarthy and Pat Hayes • Programs with Common Sense • McCarthy, later by Doug Lenat

17. 1970s: Knowledge Centric AI Understanding Speech in 70s • Blackboard Model • Hearsay • Communicating and Collaborating Knowledge Sources • Hypothesis and Verify paradigm of Knowledge Sharing • At Stanford: Penny Nii and EAF • Performance Matters: Compiling Knowledge • Dragon System • Compile Knowledge Sources into an integrated graph structure representation • Harpy • Graph optimization to eliminate redundant sub-graphs • Beam Search prunes search to look at promising alternatives and eliminates backtracking

18. Sagan Report 1978Machine Intelligence and Robotics in Space • Role of MI and Robotics within NASA • Space Exploration • Space Exploitation • Space Colonization • “Accidents Happen to Prepared Minds” (Simon quoting Pasteur) • Working on the Sagan report (as the vice-chair) prepared me for the recognizing the enormous problems of knowledge capture and use implied by the “Songs of the Distant Earth”

19. 1980s: At the Robotics Institute at CMUNecessary Conditions for Self Reproducing Factories • Self Reproducing • Experiments at RI with Fritz Prinz (now at Stanford) on Self Reproducing Lathes in 1983-84 • Self Repair • Precision remote tele-operation: McCarthy’s Proposal • Self Diagnosis • Self Awareness • Self Operation • Experiments in Mechanical MOSIS with Paul Wright (now at UC Berkeley) • The 90% Solution

20. Arthur Clarke in 1985The Songs of the Distant Earth • Colonization of Earth-like Planets in other solar systems • Capturing the Knowledge for the Mothership • 3000AD + • God-made knowledge • Man-made knowledge • Vedas, Tripetikas, Bible and Koran left behind! • Actionable Knowledge • Structured, Unstructured, Implicit Knowledge • Can we do it? What would it take? • What are the intermediate goals?

21. AI in the 80s: To be or Not to be • Creation of AAAI • Newell as President and Feigenbaum as President Elect • Bruce Buchanan, Bob Balzer, Bob Englemore… • The Ascent and Decline of AI Industry • EAF at the center • Boom Times: AAAI in Philadelphia – 6000+ people • Attacks and Self doubt • Change the Name?

22. Musings in the 1980s: What is AI anyway? • Newell’s criteria for an Integrated Intelligent System • Learn from Experience • Use Vast Amounts of Knowledge • Exhibit Goal Directed Behavior • Tolerate Error and Ambiguity • Communicate using Language and Speech, and • Operate in Real time • Laws of AI • Faced with Complexity, humans choose suboptimal solutions • Humans don’t give up claiming it is NP-Complete • An Expert knows 50K +/- 20K Chunks of Knowledge • A physical symbol system is necessary and sufficient for Intelligence • Search Compensates for Lack of Knowledge • When in doubt sprout! • Knowledge circumvents the need for Search • Knowledge reduces uncertainty eliminating trial and error behavior

23. 1997 Deep Blue beats KasparovThe Grand Challenge Problems: Knowledge is Indispensable • World Champion Chess Machine • Read a book and answer questions at the end of chapter • Observe and learn to assemble a Mars Rover or a bicycle • …..

24. Village GoogleAn AI-Centric and AI-Complete Problem • Can an uneducated person benefit from the use of Information Technology? • Can IT be affordable, accessible and available? • 4Cs: Connectivity, Computing Platform, Capacity Building, and Content • Access to Knowledge and Knowhow? • The Village Google experiment with UN-FAO • Question: Speech in local language • Answer: Video answer from “Expert” in local language • AI-Centric and AI-Complete

25. Major Breakthroughs in AI of the 20th Century • Enabled by Brute-force, Heuristics, Human Coding of Rules and Knowledge, and Simple Machine Learning (Pattern Recognition) • World Champion Chess Machine • IBM Deep Blue • Mathematical Discovery • Proof Checkers • Accident Avoiding Car • CMU: No Hands Across America • Robotics • Manufacturing Automation • Disaster Rescue Robots • Speech Recognition Systems • Dictation Machine • Computer Vision and Image Processing • Medical Image Processing • Expert Systems • Rule Based Systems • Knowledge Based Systems

26. Major Breakthroughs in AI in 21st Century • Enabled by Big Data and Machine Learning • Language Translation • Google Translate: Any Language to Any Language • Speech to Speech Dialog • Siri, Cortana, Alexa • Autonomous Vehicles • CMU, Stanford, Google, Tesla • Deep Question Answering • IBM’s Watson • RoboSoccer • World Champion Poker • CMU Libratus • No Limit Texas Hold’em Poker

27. The Next Millennium:Research Agenda for the Future of AI? • Improve Human Productivity by 1000% • 10 – 20 Years • Grand Challenge Problems • 20 – 50 Years • Human Level AI • 50 – 100 Years • Super Human AI • 100 – 1000 Years

28. Looking back: What we missed! • Personal Computers! • Alan Kay’s dynabook vs Apple and PCs • Internet and the WWW • ARPAnet in 1968 with Stanford as one of the initial nodes • Moore’s Law and VLSI • Graphics • Human Computer Interaction • UI design

29. Looking back: off in timing! • Speech • Vision • Robotics • Natural Language

30. Recent Trends in AI • Learning Systems • Learn from examples • Learn from experience • Dynamic Learning • Learning from Sparse data • Architecture of Intelligence • Integrated Intelligence • Learn from Experience • Use Knowledge • Communicate using Speech and Language • Operate in real time • etc

31. Recent Trends in CS • Lisp → Functional Languages • Timesharing → Thin Clients • Algorithm Design → Scalable Dependable • Systems → beyond OS • Graphics → 2D to 3D • UI → Illiterate users? • Hardware → Low power mobile

32. Whither AI? • Arthur Clarke’s The Songs of the Distant Earth • Ray Kurzweil’s Immortality

33. Whither CS? • Computers are for Entertainment and Communication • Not for Computing • “People are the Killer App” from Parc • Software as Service • Death of Software Product Market • Net 2.0 and Web Services • Cell Phone as the Dominant Computing Platform • Embedded Body Computers

34. In Conclusion… • Much of what transpired in AI and CS in the last 40 years can be seen to have roots in the activities of the 60s! • Except that we now have a million times more computing power! • AI Needs Robust CS • Million times more processing • Million times more Memory • Million times more Bandwidth • May be we do need 1.7 Einsteins, 3 Maxwells and 0.7 Manhattan project (McCarthy, 1980s) to get there