The Engaging User Experience & the Natural User Interface

1. The Engaging User Experience & the Natural User Interface • Tim Huckaby • InterKnowlogy Chairman/Founder • Actus Interactive Software Founder & CEO

2. A little bit about me and why I am here… The Engaging User Experience & the Natural User Interface

3. About Tim, InterKnowlogy & Actus… • InterKnowlogy(www.InterKnowlogy.com) • Tim Huckaby, Founder/Chairman (TimHuck@InterKnowlogy.com) • 13+ year old services firm – Custom Application Development • focused on building NUI solutions • Web, mobile, and desktop • Windows 7/8, PixelSense, WP7/8, Azure, Kinect, XAML stack • - Largest Client: Microsoft • Actus Interactive Software (www.Actus-Software.com) • Tim Huckaby, Founder/CEO (TimHuck@Actus-Software.com) • Software Product Company • Interactive Kiosk and Active Digital Signage • Multi-Touch, Gesture, and Voice Driven Solutions The Engaging User Experience & the Natural User Interface

4. Where We have Been and Where we are Going The 4 Manifestations of NUI: Multi-Touch, Vocal, Gesture, Neural There are a number of tenants of a great user interface that will last the test of time NUI enabled applications improve the communication, education and user experience across the software continuum.  Demos, Demos, Demos…. Objectives And Agenda The Natural User Interface (NUI) The Engaging User Experience & the Natural User Interface

5. Gesture Capable • Motion Based Interface • Tracking of full-body movement, facial expression, and voice with precision • Voice Capable • Multi-array Microphone • Uniquely Distinguish Audio & Voices • Nueral Capable • Conscious and non-conscience interfaces • Non Invasive • Multi-Touch Capable • Common Gestures of Touch • Physical Object Interaction • Virtual Object Interaction NUI: The Natural User Interface The Engaging User Experience & the Natural User Interface

6. January 27, 2010 The Engaging User Experience & the Natural User Interface 6

7. The Engaging User Experience & the Natural User Interface

8. We have come a long way… The “Green Screen” The Engaging User Experience & the Natural User Interface 8 8

9. Image Courtesy of Computer History Museum The Engaging User Experience & the Natural User Interface

10. Touch Capable Computing Devices Are Not New Apple iPad Windows 8 - Metro 1972: PLATO IV Touch Screen Terminal The Samsung SUR40 for Microsoft PixelSense Past Present Future (now) The Engaging User Experience & the Natural User Interface

11. What is Relatively New is: • Inexpensive Touch Capable, Non Proprietary Hardware • A Operating Systems designed for Multi-Touch • A Consistent Multi-Touch API for the major Hardware/Software Platforms • Developer Productivity in Touch Capable Apps • The Forecast of Multi-Touch Hardware Capability in Every New Computing Device in the Short Term The Engaging User Experience & the Natural User Interface

12. Demo The “Cancer App” The “Autocad App” Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

13. Video Grey’s Anatomy Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

14. Video Gesture Heart Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

15. EMN8 Demo Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

16. Defines a set of low-level events that represent one or more points of contact with a touch-sensitive surface Addresses pen-tablet devices, such as drawing tablets, with consideration toward stylus capabilities. Changes of those points with respect to the surface and any DOM elements displayed upon it (e.g. for touch screens) or associated with it (e.g. for drawing tablets without displays). HTML 5 Touch Events, Version 1W3C Working Draft 24 January 2013 “This specification is the 24 January 2013 Last Call Working Draft. The comment period for this Last Call Working Draft ends 14 February 2013.” Source: https://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html The Engaging User Experience & the Natural User Interface

17. The Engaging User Experience & the Natural User Interface

18. The Engaging User Experience & the Natural User Interface

19. Demo Democratic National Convention Voting App Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

20. The Tenants of an Engaging UX Use the Power of Faces People love faces; We see faces where they don’t exist We have DNA with the sole purpose of recognizing faces Take advantage of this by putting faces on your App Have the faces to look at important part of your App People instinctively follow the eyes of the face. Use Food, Sex and Danger When someone sees something new, they place it into one of four categories: Can I Eat it? Can it Eat Me? Can I Mate with it? …and everything else. If you want your App to be engaged and interesting, put it in one of the first three categories. The Engaging User Experience & the Natural User Interface

21. The Tenants of an Engaging UX Tell a story We learn through stories. It’s how we teach our children. It’s the Bible; It’s what we see in movies. Stories are a big part of how information is conveyed to us. If you have information to give to the user, put it in story form. It doesn’t matter what medium you use (words, pictures, music), but using a story narrative will help your user understand and retain your information. Build commitment over time Let the user choose how they want to interact with you (RSS, Twitter, Facebook) Make sure that you don’t take advantage of the trust that is implicit in that interaction. Use Natural Interactions of Objects Natural Movements Fluidity Animate / Exaggerate: Slow things down so the eye can follow The Engaging User Experience & the Natural User Interface

22. What is also Relatively New:User Experience with Naturally Behaving Objects • Large Form Factors demand natural behaviors • The User Experience is “never done” • It can always be improved • Tablet Computing paved the way • Software Physics Engines are readily available in open source The Engaging User Experience & the Natural User Interface 22

23. Fluid LayoutPixelsence Craps Demo Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

24. “Game-ification” User Experiences that Enhance Learning Learning made Fun / Entertaining Proven as a learning tool beyond just Children Computer based Games are a significant part of popular culture Educational Video Game Curriculums • MIT, UC Irvine, Minnesota, Michigan State, etc. The Engaging User Experience & the Natural User Interface

25. NASA Marsbound Demo Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

26. World's Fastest-Selling Consumer Electronics Device … a Future Beyond Gaming Combination of RGB camera, depth sensor and multi-array microphone RGB camera delivers three basic color components Depth sensors “sees” in 3-D Microphone locates voices by sound and excludes ambient noise Consumer Priced: $249.99 Tracks full-body movement, facial expression, and voice with precision Kinect sensor bar scans and monitors 1 million points on the body Kinect for Windows SDK Skeletal Tracking Face, Gesture Recognition Audio Echo cancellation Audio Beam Forming Speech Recognition Machine Learning Kinect for Windows The Engaging User Experience & the Natural User Interface

27. Kinect for Windows:Actus Interactive Digital Content System (Kiosk)IK Physical TherapyIK Touch-less Operating Room Demo Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

28. Kinect for Windows Competitors Leap Motion • $70 • https://leapmotion.com • Not available yet SoftKinetic • http://www.softkinetic.com • $249 • Distance and Near Mode • Multiple Products • In the space for over a decade Panasonic • Works outside in broad daylight • $2,600 The Engaging User Experience & the Natural User Interface

29. Neural Interfaces:Conscious and Non-Conscious High fidelity EG acquisition device No conductive gel, etc. Non-invasive Use Case Examples: • Conscious: mind manipulation of objects • Non-Conscious: Ranking music • First target market: gaming • $300!!! The Engaging User Experience & the Natural User Interface

30. Video Emotiv Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

31. We are within a decade of building the Holodeck“The Ultimate NUI Application” The Engaging User Experience & the Natural User Interface

32. Video HoloDesk - Direct 3D Interactions with a Situated See-Through Display Tim Huckaby • Founder, InterKnowlogy & Actus Software • Microsoft RD & MVP The Engaging User Experience & the Natural User Interface

33. We are at the cusp of a revolution in software & hardware  Within a realistic time frame NUI computing will be the norm There are a number of design tenants that can enhance User Experience The concepts of NUI change the way we design and build software Using Natural User Interface (NUI) technologies improve the User Experience The hardware, tools, plumbing and platforms for NUI computing are powerful and available now.…and will just continue to get better, cheaper, powerful and more available Summary The Engaging User Experience & the Natural User Interface

34. Tim Huckaby More info on InterKnowlogy: • www.InterKnowlogy.com • Silverlight.InterKnowlogy.com • www.vimeo.com/InterKnowlogy More info on Actus Software: • www.Actus-Software.com About Tim Huckaby… • Founder/Chairman, InterKnowlogy • Founder/Chairman, Actus Software • Microsoft® Regional Director – Southern California • Microsoft® .NET Partner Advisory Council Founder / Member • Microsoft® MVP – Kinect for Windows • INETA Speaker – International .NET Users Group Association • Board Member: Association for Competitive Technology • Board Member: DevIntersection • Board Member: Microsoft Healthcare Partner Advisory Board • Author / Speaker • Contact me:E-mail:TimHuck@InterKnowlogy.com • TimH@Actus-Software.com • Phone: • Office: 760-444-8640 • Mobile:: 619 990 9200 • Blog: • http://team.interknowlogy.com/blogs/timhuckaby • FaceBook: • http://www.facebook.com/Tim.Huckaby.Carlsbad • LinkedIn: • http://www.linkedin.com/in/timhuckaby • Twitter: @TimHuckaby The Engaging User Experience & the Natural User Interface