New Developments in Large Data that have Immediate Application in Industry

1. New Developments in Large Datathat have Immediate Applicationin Industry (but you haven’t heard of yet) Joseph Turian @turian MetaOptimize #bbuzz

2. perhaps you should close your laptops

3. How do you get a competitiveadvantage with data?

4. How do you get a competitiveadvantage with data? • More data

5. How do you get a competitiveadvantage with data? • More data • Better algorithms

6. When big data gives diminishing returns,you need better algorithms

7. When should you use better algorithms?

8. When should you use better algorithms? • If they are really cool algorithms

9. When should you use better algorithms? • If they are really cool algorithms

10. When should you use better algorithms? • If they are really cool algorithms • If you have a lot of time on your hands

11. When should you use better algorithms? • If they are really cool algorithms • If you have a lot of time on your hands

12. Only use better algorithms if they will qualitatively improve your product

13. Who am I?

14. Who am I? • Engineer with 20 years coding experience • Ph.D. 10 yrs exp in large-scale ML + NLP

15. What is MetaOptimize?

16. What is MetaOptimize? optimizing the process of

17. What is MetaOptimize? optimizing the process of optimizing the process of

18. What is MetaOptimize? optimizing the process of optimizing the process of optimizing the process of optimizing the process of optimizing the process of optimizing the process of optimizing the process of optimizing the process of optimizing the process

19. What is MetaOptimize? • Consultancy on: • Large scale ML + NLP • Well-engineered solutions

20. “Both NLP and ML have a lot of folk wisdom about what works and what doesn't. [This site] is crucial for sharing this collective knowledge.” - @aria42 http://metaoptimize.com/qa/

21. Outline • Deep Learning • Semantic Hashing • Graph parallelism • Unsupervised semantic parsing

22. Outline • Deep Learning • Semantic Hashing • Graph parallelism • Unsupervised semantic parsing

23. Opportunity with Deep Learning • Machine learning that’s • Large-scale (>1B examples) • Can use all sorts of data • General purpose • Highly accurate

24. Deep Learning

25. Deep Learning • Artificial intelligence???

26. Natural Intelligence

27. Natural Intelligence Works!

28. Artificial Intelligence

29. Artificial Intelligence • Still far from the goal! • Why?

30. Where does intelligence come from?

31. Intelligence comes from knowledge

32. How can a machine get knowledge? Human input

33. NO!

34. Intelligence comes from knowledge.Knowledge comes from learning.

35. Statistical Learning • New multi-disciplinary field • Numerous applications

36. Memorize? Generalize? or • Mathematically: fundamentally difficult • Easier for humans • Easy for machines • Harder for humans

37. How do we builda learning machine?

38. … … Deep learning architecture Output: is bob? Highest-level features: Faces Abstract features: Shapes Primitive features: Edges Input: Raw pixels … … …

39. … … Shallow learning architecture …

40. Why deep architectures?

41. subsubsub2 subsubsub1 “Deep” computer program subsubsub3 subsub1 subsub2 subsub3 sub1 sub2 sub3 main

42. “Shallow” computer program subroutine1 includes subsub1 code and subsub2 code and subsubsub1 code subroutine2 includes subsub2 code and subsub3 code and subsubsub3 code and … main

43. “Deep” circuit

44. “Shallow” circuit output … 2n … 1 2 3 n input

45. Insufficient Depth Insufficient depth = May require exponential-size architecture Sufficient depth = Compact representation … … … … … 2n 1 2 3 O(n) … … 1 2 3 n 1 2 3 n

46. What’s wrong with a fat architecture?

47. Overfitting! bad generalization

48. Occam’s Razor

49. Other motivations for deep architectures?

50. Learning Brains • 1011 neurons, 1014 synapses • Complex neural network • Learning: modify synapses