Deep Learning Revolutionizes Artificial Intelligence Development

1. Artificial Intelligence – Deep Learning and its Applications 人工智能 – 深度學習及其應用 Ir Dr F C Chan Past President The Hong Kong Institution of Engineers 10 May 2018

2. Artificial Intelligence – Deep Learning and its applications • AlphaGo • Artificial Intelligence Development • Deep Learning • Features Extraction • Applications • Conclusion

3. AlphaGo Lee Vs Lee Sedol 李世石 李世石 黃士傑 AlphaGo won by 4-1 AlphaGo A combination of neural network and algorithm 9 and 15 March 2016

4. AlphaGo Master Google hosted in Wuzhen in China 23-27 May 2017 Vs Ke Jie 中國棋王柯潔 AlphaGo won by 3-0 Google has announced that the research team behind AlphaGo would now focus their energy into tackling more complex issues and develop highly advanced algorithms to tackle some of the most complex problems in the world.

5. AlphaGo Zero Elo ratings - a measure of the relative skill levels of players in competitive games such as Go - show how AlphaGo has become progressively stronger during its development

6. AlphaGo Zero Vs Deep Blue • Deep Blue(IBM) beat Word Chess Champion • Garry Kasparov in 1997 • Won 3.5 vs 2.5 in a match of 6-game • Go Vs Chess • Bigger board, 19x19 vs 8x8 • Simpler in rules, more move possibilities, • 361 vs 28 • Longer game, 150 vs 80 moves • Alpha Go vs Deep Blue • Deep Blue can only play chess • Alpha Go is general purpose, can win 49 different arcade games AlphaGo Zero No Built-in expert knowledge Creativity and Intuitive insights Deep Learning can revolutionize everything

7. AI Development • Artificial Intelligence (1950s) – Giving intelligence to machine • Machine Learning (1980s) – realizing artificial intelligence (speech recognition, image recognition, playing go, dialogue) • Deep Learning (2006) – for machine learning for higher prediction accuracy • A powerful class of machine learning model • Modern reincarnation of artificial neural network • Collection of simple, trainable mathematical functions

8. AI Development 1958: Rosenblatt’s Perceptron algorithm 1969: Minksky showed Perceptron could not solve the XOR problem, connectedness, parity. 1986: Rumelhart developed Backpropagtion algorithm to train neural network Mid 90’s: Cortes and Vapnik published paper on Support Vector Machines 2006: Hinton and Salakhutdinov proposed using Restricted Blotzmann Machine for pre-train Deep Neural Network

9. AI Development 2007: Fei-Fei Li’s ImageNet assembling a databse of 14 million labled images (Data drives learning) 2011: Micorsoft explored Speech recognition and IBM’s Watson 2014: Google acquired DeepMind, combing deep learning and reinforcement learning 2016: DeepMind’s AlphaGo defeated world champion Lee Sedol

10. Deep Learning applications Google: 2011 launched deep-learning focused project. 2014 bought DeepMind with AlphaGo. Microsoft: strong in speech-recognition and translation Facebook: translate user posts in more than 40 languages Baidu: for speech recognition, translation, photo search and self-driving car project

11. Perceptron Human mindset: Axons send signals to other cells while Dendrites receive them. Perceptron approach

12. Perceptron • Activation Function Network • Linear • Binary • Sigmoid • Stochastic Binary

13. Perceptron Logistic Regression

14. Handling XOR

15. Neutral Network Weights are the important elements in Neural Network formulation

16. Neutral Network Example: 4- layer network with 2 output units

17. Deep Learning Rules Based System Hand designed Program Input Output Classic Machine Learning Hand designed Features Mapping from Features Input Output Deep Learning Additional Layers of more Abstract Features Mapping from Features Sample Features Output Input

18. Deep Learning

19. Deep Learning

20. Neutral Network • Deep Learning • Nonlinear activation function • Neuron could connect to every input • Multi-levels with millions neurons • Any function can be computed • The drive behind Deep Learning • Faster machines and core (CPU/GPU) • Big data (with large dataset) • New models and algorithms Neural Network: pattern recognition or data classification

21. Deep Learning

22. Deep Learning • Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques • Adaptive learning • Self Organization • Fault Tolerance

23. Deep Learning Implementation • Cloud with Big Data • Google, Facebook, Amazon, Baidu, Tencent • PC Implementation • Linux cluster with GPU servers • Python • TensorFlow, Theano, Keras

24. Deep Learning Software

25. Artificial Intelligence – Deep Learning and its applications Trend Prediction Recognition New Knowledge Making Sense Replacing Human

26. Artificial Intelligence – Deep Learning and its applications Information retrieval (search engines) Pattern recognition Audience targeting Sentiment analysis (based on written text) Personalization Automation Natural Language Processing Social media mining Organic search and content performance Brand and product differentiation

27. Artificial Intelligence – Deep Learning and its applications • Language Translation • Speech Recognition • Generating Handwriting • Face Recognition • Autonomous Driving • Generating Arts • Imitating Famous Painters • Generating Music • Generating Photos

28. Deep Learning Applications Power Systems Peak Load Forecast (e.g. Maximum Demand) Failure Prediction (e.g. Battery) Condition Monitoring (e.g. Partial Discharge) …..

29. Deep Learning Applications Data Centres Increase its energy efficiency by 15% 120 different variables (sensors, temperatue gauges, fans, windows…)

30. Conclusion Artificial Intelligence – Deep Learning will have major impact to our living Specialized applications in various domains

31. Artificial Intelligence – Deep Learning and its Applications Ir Dr F C Chan Past President The Hong Kong Institution of Engineers Thank you