470 likes | 856 Views
PC17. Developing For Microsoft Surface. Brad Carpenter General Manager Surface Platform. Agenda. Microsoft Surface and the future of computing How you can be part of this revolution How the Surface SDK enables rapid innovation Microsoft’s multi-touch roadmap Questions and Answer.
E N D
PC17 Developing ForMicrosoft Surface Brad Carpenter General Manager Surface Platform
Agenda • Microsoft Surface and the future of computing • How youcan be part of this revolution • How the Surface SDK enables rapid innovation • Microsoft’s multi-touch roadmap • Questions and Answer
Interface Paradigm Shift CLI GUI NUI
Microsoft Surface And NUI Microsoft Surface computing uses sensing and display technology to infuse everyday surfaces with digital content and is comprised of the following unique attributes: Object recognition Direct interaction Multi-touch Multi-user Object recognition Direct interaction Multi-touch Multi-user
How Surface Is Being Used Today • 5 Verticals • Automotive • Financial Services • Healthcare • Leisure and • Entertainment • Retail
announcing… Microsoft Surface Hardware And SDK availability
Developing ForMicrosoft Surface Robert Levy Program Manager Surface SDK Doug Kramer Lead Developer Surface SDK
demo Visualizing Surface Input
Congratulations! You already know the basics of building applications for Surface
Surface v1 Architecture Surface Applications WPF APIs Core API Shell UI & APIs Vision System Windows Integration Windows Vista SP1 Microsoft Surface Hardware & Drivers
Surface And WPF • Expose Surface capabilities in a way consistent with WPF • Infrastructure integration for consistent behavior with mouse • Automatic hit testing to find the control being touched • Sets attached properties on controls (IsContactOver, etc.) • Raises routed events on controls (ContactDown, etc.) • Custom versions of most built-in WPF controls • Custom controls for Surface-specific UX • Base classes for building custom Surface controls
Surface API Usage With WPF XAML <Window> <Canvas Mouse.MouseDown=“OnMouseDown“> <Button Click=“OnButtonClick”/> </Canvas> </Window> Code void OnMouseDown(object sender, MouseEventArgs e) { // ... } void OnButtonClick(object sender, RoutedEventArgs e) { // ... } • XAML • <s:SurfaceWindow> • <Canvas s:Contacts.ContactDown=“OnContactDown“> • <s:SurfaceButton Click=“OnButtonClick”/> • </Canvas> • </s:SurfaceWindow> • Code • void OnContactDown(object sender, ContactEventArgs e) • { // ... • } • void OnButtonClick(object sender, • RoutedEventArgs e) • { // ... • }
Surface Classes For WPF Input Mouse Stylus MouseEventArgs StylusEventArgs Tablet TabletDevice MouseDevice StylusDevice
Surface Versions Of Common Controls SurfaceWindow Full screen on the Surface Oriented towards the user SurfaceMenu SurfaceContextMenu SurfaceMenuItem Multiple menus can be used at the same time SurfaceButton SurfaceRadioButton SurfaceCheckBox SurfaceToggleButton Only “click” when all contacts are removed
Surface Versions Of Common Controls • SurfaceListBox • SurfaceScrollViewer • Determines scrolling vs. select • Allows flicking • Provides “elastic” effect • SurfaceSlider • SurfaceScrollBar • Allows flicking • Track input results in smooth animation and capture • Subtle UX when not in use • SurfaceTextBox • SurfacePasswordBox • Invokes virtual keyboard oriented towards user when touched
Neat, But Does It Justify Buying A Surface? • We’ve made it easy to port WPF and XNA apps to Surface • Great for developers • Re-use code, skills, and tools • Get started quickly • …but little value to users
Is this really just a big PC? I don’t like having to share and wait my turn Constantly reaching across the table is not fun Reading upside down makes my head hurt
Making Surface Shine • Great apps leverage all of what makes Microsoft Surface unique • Multi-touch • Multi-user • 360° UI • Gestures and manipulations • Interaction with physical objects • WPF and the Surface SDK make this easy
Introducing The ScatterView Control • Quickly enables 360°, multi-user apps • Gives users direct, shared control of layout • Standardizes manipulations for Resize, Move, and Rotate • Same familiar programming model as ItemsControls, ListBox, TreeView, Menu, etc.
demo ScatterViewIn Action
Recognizing Tagged Objects Byte Tags Contact.Tag.Byte 256 unique values Contact.Tag.Identity >340,282,366,920,938,000,000,000,000,000,000,000,000unique values Identity Tags
Introducing The TagVisualizer Control Robert Levy Send Email Simplifies common scenarios involving tagged objects • Displays interactive UI under, around, or next to objects • Keeps UI in sync with movements • Handles temporary lifting TagVisualizer Control UI for each object Tag values to look for Positioning of UI relative to tag Max # of each object allowed How to handle tag removal
demo TagVisualizerIn Action
Is this really just a big PC? I don’t like having to share and wait my turn Constantly reaching across the table is not fun Reading upside down makes my head hurt
Controls For Making Surface Shine Robert Levy Send Email ScatterView • Multi-touch • Multi-user • 360° UI • Natural interactions with virtual objects TagVisualizer • Object recognition • Natural interactions with physical objects
demo Redesigning An App For Surface
Touch Development Roadmap Windows 7 Release NET 3.5 / Surface 1.0 Release NET 4.0 / Surface 2.0 Release WPF Application Native Win32 Application WinFormsApplication Surface Application Surface SDK 1.0 Surface SDK 2.0 Multi-Touch Controls Surface Multi-Touch Controls & API Multi-Touch API Managed Wrapper and Interop WPF 3.5 x WPF 4.0 WPF 3.5 Multi-Touch API Multi-Touch API and Controls Surface Hardware Windows Vista Surface Hardware Windows 7 Windows 7 Multi-Touch API
Summary • Microsoft Surface is shaping the future • Multi-touch, multi-user, 360° UI, object recognition, … • UI design requires creativity and innovation • Coding builds upon what you already know • Microsoft is creating a unified API for touch
Call To Action • Help us change the world • Create apps using Surface’s unique capabilities • Get Microsoft Surface developer hardware • Order forms on http://surface.com • $1,500 discount until November 15th • Swipe your badge to receive an e-mail invitation to join http://community.surface.com • Download the Microsoft Surface SDK & Simulator • Participate in online forums and training
Related Content • Hands on Labs • PCHOL19: Using the Microsoft Surface Simulator • PCHOL20: A Lap Around the Microsoft Surface SDK • Surface HOL: Exploring Vision Based Object Recognition with Microsoft Surface • Demo applications from partners • Scavenger Hunt game • PC03: Developing multi-touch applications with Windows 7 • Wednesday, 1:15
Evals & Recordings Please fill out your evaluation for this session at: This session will be available as a recording at: www.microsoftpdc.com
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Simulator for Off-Table Dev & Test • Replicates input • With multiple mice • With one mouse • Click to place multiple contacts • Click and drag to group them together • Move, rotate, and stretch the group • Supports all types of contacts • Finger, resizable blobs, and tags • Supports input automation for testing • Recording and playback UI • Simulator APIs 37
Manipulation & Inertia Processor APIs • Low level APIs for building controls with consistent behavior • Reduces the cost of building new controls • Increases consistency for end users • Interpreting Surface input as 2D transformations • Input: list of contacts to track • Output: scale, rotation, movement, velocity data (~60 FPS) • Used by: ScatterView, ScrollViewer, Slider, Concierge map • Two modes of inertia • Initial velocity + deceleration rate • Realistic • Used by ScatterView and Concierge map • Initial velocity + final position/orientation • Goal-based • Used by ScrollViewer and Slider
Ups & Downs Mouse.MouseDownMouse.MouseUp • Contacts.ContactDownContacts.ContactUp
Up & Down Event Comparison • Similarities: • Naming convention & meaning • Routing strategy • Tunneling (“preview”) events & bubbling events • Events stop once marked as handled • Hit testing & capture logic • Differences: • Need to find out which contact the event is for • New Contact object is created with each Down event
Movements Mouse.MouseMove … Mouse.MouseMove … Contacts.ContactChanged … Contacts.ContactChanged …
MouseMoved == ContactChanged? ContactChanged includes: • Position • Orientation • Recognized type • Size • Shape • Area Wait a sec… he said this was supposed to be the same as mouse. Why are these events named differently?
Movement Event Comparison • Similarities: • Routing strategy • Tunneling (“preview”) events & bubbling events • Events stop once marked as handled • Hit testing & capture logic • Event for changes in position • Differences: • ContactChanged is not just for movements • Change events only happen between Down & Up • Need to find out which contact the event is for
Position Contact.GetPosition(UIElement) • Relative to specified UI element • GetPosition(window) = {50, 100} • GetPosition(button) = {10, 10} • To track position changes across multiplecontrols, get position relative to a common parent
Orientation 270° Contact.GetOrientation(UIElement) • Degrees clockwise from relative east • Range and usefulness varies • Tagged objects: 360° • Generic blobs: 180° • Fingers: 360° but…unreliable when straight down 0° 180° 90°
Ellipse & Bounds RectContact.BoundingRect Ellipse Contact.GetEllipse(UIElement) void Contact.UpdateEllipse(Ellipse, UIElement) • Ellipse is returned with transforms that counteract transforms on the specified element • Adding the shape as a child element results in UI matching the physical object • Bounding rectangle is always screen-aligned • UpdateEllipse method can improve performance