Multi-Touch on Microsoft Surface and Windows 7 for .NET Developers

1. CL27 Multi-Touch on Microsoft Surface and Windows 7 for .NET Developers Anson Tsao Lead Program Manager WPF Robert Levy Program Manager Surface

2. Win one of thirty $50gift card!!!Complete a quick survey atwww.WontTakeLong.com

3. Agenda • Introduction • Multi-touch in WPF 4 & SL 3 • Raw Input APIs • Manipulation/Inertia APIs • Controls • Microsoft Surface: Beyond Touch • Controls • Unique capabilities • Roadmap for platform alignment • Obligatory slides • Q&A

4. Multi-Touch… • …creates new opportunities. • Software that is intuitive for users to walk up & master • Software that is collaborative with multiple users • …is not new. • Researchers have been exploring this stuff for decades • …is now going mainstream. • Hardware: Robust touch-capable hardware from Microsoft and OEMs • Dozens of Win7 touch devices now available • OS: Windows 7 has great touch support baked in • SDKs: Microsoft is making touch easy to leverage • You: Developers & designers creating innovative apps that take advantage of the technology

5. Microsoft Surface & WPF • With Surface, Microsoft became a thought leader in the multi-touch arena • Surface has been shipping a multi-touch dev platform for several years • Built as a set of extensions to WPF 3.5 • Touch APIs from Surface are being baked into WPF 4 for cross-platform use

6. WPF 4 Touch Capabilities Anson Tsao

7. WPF 4 Multi-Touch APIs Overview

8. Raw Multi-Touch Events demo

9. Multi-Touch on the Web with Silverlight • Subset of the WPF4 touch input APIs • Reports groups of touch events in “frames” • Apps are responsible for hit testing and event routing • Manipulation & Inertia Processor API will be available on surface.com • Same algorithms used by Surface, WPF, & Win7

10. Silverlight 3 Touch APIs - Listening • public partial class MainPage : UserControl • { • public MainPage() • { • ... • // listen to touch events from the system • Touch.FrameReported += • new TouchFrameEventHandler(OnFrame); • } • void OnFrame(object sender, TouchFrameEventArgs e) • { • // enumerate and respond to touch events • } • }

11. Silverlight 3 Touch APIs - Processing • void Touch_FrameReported(object sender, TouchFrameEventArgs e) • { • TouchPointCollectiontouchPoints = e.GetTouchPoints(LayoutRoot); • foreach (TouchPointtp in touchPoints) • { • if (tp.Action == TouchAction.Down) { • // a new touch has come down • } • if (tp.Action == TouchAction.Move) { • // a previously down touch has moved • } • if (tp.Action == TouchAction.Up) { • // a touch has been removed • } • } • }

12. Multi-Touch Manipulations demo

13. Manipulation Container IsManipulationEnabled=true IsManipulationEnabled=true Canvas • Handle ManipulationStarting Event • Set Manipulation Container Manipulation Container • Handle ManipulationDelta Event • Coordinates relative to Manipulation Container Update RenderTranform

14. Manipulation Events • Initialize: • Mode • Container • Pivot StartInertia() StartInertia() Completed()

15. Manipulation Inertia demo

16. Inertia Behaviors • Deceleration and Displacement/Rotation/Expansion mutually exclusive • Deceleration – useful to simulate friction • Displacement – useful on page flips • Default – no inertial movements

17. Touch Panning Demo demo

18. Touch Panning Support • In WPF4 Release Candidate Builds (not Beta2) • Cancel() method on manipulation events • Cancelled manipulation promoted to mouse • ScrollViewer handles manipulations • PanningMode • None, Both, Horizontal/VerticalOnly, Horizontal/VerticalFirst • PanningDeceleration • PanningRatio • Styling changes to intrinsic controls

19. Microsoft Surface’s Extensions of WPF Robert Levy

20. Surface WPF controls make touch UI easy • Designed for multi-touch • Designed for simultaneous use • Designed for manipulations & inertia

21. Drawing with Surface Controls • <s:SurfaceInkCanvas/>

22. Manipulating Photos with Surface Controls • <s:ScatterView Name=“scatter"> • <s:ScatterView.ItemTemplate> • <DataTemplate> • <Image Source="{Binding}"/> • </DataTemplate> • </s:ScatterView.ItemTemplate> • </s:ScatterView> • scatter.ItemsSource =Directory.GetFiles(…);

23. Inking on Photos with Surface Controls • <s:SurfaceScrollViewer> • <s:ScatterView Name=“scatter"> • <s:ScatterView.ItemTemplate> • <DataTemplate> • <Grid> • <Image Source="{Binding}"/> • <Viewbox Margin="50"> • <s:SurfaceInkCanvas/> • </Viewbox> • </Grid> • </DataTemplate> • </s:ScatterView.ItemTemplate> • </s:ScatterView> • </s:SurfaceScrollViewer> • scatter.ItemsSource = Directory.GetFiles(…);

24. Inkable Photos on Surface demo

25. Surface Toolkit for Windows Touch announcing Controls, samples, templates, and docs Coming shortly after the WPF4 launch

26. Surface SDK Features for Multi-Touch • Common controls optimized for Multi-Touch • Controls design primarily for Multi-Touch • Essential Multi-Touch UX Functionality

27. Inkable Photos on Windows 7 demo

28. Surface is not just multi-touch • Windows 7 & WPF 4 are a great baseline for touch • Surface creates unique opportunities with specialized HW • Robust top • Horizontal form factor • Infrared camera array

29. Surface: Beyond Multi-Touch • Massive multi-touch • Multi-user experiences • Object recognition • Optical engineering

30. Surface: Multi-User • Trend: Software continues to reduce the need for face-to-face human interactions • Problem: Happiness and creativity depend on social interactions • Surface brings people together • Industrial design encourages users to gather • Input capabilities allow simultaneous use • Apps facilitate collaboration / competition

31. Surface: Massive Multi-Touch • Surface can efficiently track 50+ simultaneous fingers and objects

32. Surface: Objects • Uniquely identify & react to billions of tagged objects Byte Tags 256 unique values Identity Tags >340,282,366,920,938,000,000,000,000,000,000,000,000unique values • Tags include orientation indictors dots

33. Surface: Optics • Determine the size and shape of every input • See into and project onto physical objects

34. Optical Magic with Surface demo surface.com/monster

35. Optical Magic with Surface demo Behind the scenes video at: surface.com/monster

38. Surface Application Showcase video

39. Surface Application Showcase video

40. Surface vNext Integration & Extension of WPF 4 • All input is routed through the WPF InputManager • Surface input provider can send to multiple windows • Surface extension methods provide access to additional input data (tag values, orientation, shape, etc) Windows Touch input provider Manipulation & Touch Events WPF InputManager Apps & Controls Surface input provider Surface extension methods

41. Getting Surface Data from WPF Extensions • void OnTouchDown( object sender, TouchEventArgs e ) • { • // built in to WPF • point p = e.Touch.GetPosition(this); • // calling Surface extension directly • TagData t1 = Microsoft.Surface.TouchExtensions.GetTagData(e.Touch); • // calling Surface extension w/ C# magic • TagData t2 = e.Touch.GetTagData(); • // some other Surface extensions • double o = e.Touch.GetOrientation(this); • Ellipse e = e.Touch.GetEllipse(this); • }

42. Touch Development Roadmap Windows 7 (2009) Native Win32 Application WinFormsApplication WPF Application Surface Application Surface SDK 1.0 Multi-Touch Controls Multi-Touch Controls & API WPF 3.5 SP1 Managed Wrapper and Interop Surface Hardware Windows Vista Windows 7 Multi-Touch API

43. Touch Development Roadmap .NET 4.0 & Surface Toolkit (Q1 2010) Native Win32 Application WinFormsApplication WPF Application Surface Application Surface SDK 1.0 Surface Multi-Touch Controls &API Multi-Touch Controls & API Multi-Touch Controls Surface Toolkit for Windows Touch Managed Wrapper and Interop WPF 3.5 SP1 WPF 4.0 w/ Multi Touch APIs Surface Hardware Windows Vista Windows 7 Multi-Touch API

44. Touch Development Roadmap Surface vNext(Future) Native Win32 Application WinFormsApplication WPF Application Surface Application Surface SDK vNext Surface Multi-Touch Controls &API Multi-Touch Controls Surface-specific Controls & API Surface Toolkit for Windows Touch Managed Wrapper and Interop WPF 4.0 w/ Multi Touch APIs Surface Hardware Windows 7 Windows 7 Multi-Touch API

45. What’s next for Surface hardware? • Things we’re exploring: • Cheaper • Thinner • Vertical • Stay tuned…

46. Call to Action • Windows 7 • Download Silverlight 3 RTM or Silverlight 4 Beta 1 • Download Visual Studio 2010 Beta 2 w/ WPF 4.0 • Surface • By Invitation Only: http://community.surface.com • Get the Surface SDK & Simulator • Check out the samples, docs, and templates • Browse and participate in the online forums Purchase Surface hardware from Surface.com(starting at $12,500) SDK & resources now available free to everyone! Integrated with TechNet & MSDN for downloads, online docs, and forums. Start at our portal: http://surface.com/developer NEW!

47. Related Resources • Windows Touch Deep Dive (Session CL17) • WPF 4 Resources: http://connect.microsoft.com/wpf • Surface Resources: http://surface.com • Infragistics.com – free controls for Surface v1! • Map, Chart, and Carousel • CTP now available • NSquaredSolutions.com/SurfaceBook • Coming soon!

48. Win one of thirty $50gift card!!!www.WontTakeLong.com

49. YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation forms online at MicrosoftPDC.com

50. Learn More On Channel 9 • Expand your PDC experience through Channel 9 • Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses channel9.msdn.com/learn Built by Developers for Developers….