Windows Phone 8 Sensors

1. Windows Phone 8 Sensors Matthias Shapiro mashapir@microsoft.com http://matthiasshapiro.com

2. Windows Phone 8 Sensors

3. Windows Phone 8 Sensors • Motion APIs • Accelerometer • Compass • Gyroscope (Enhanced Motion) • Voice over IP (VoIP)

4. Windows Phone 8 Sensors

5. New Windows Phone Runtime Location API • Replaces Bing Maps • Supports Managed and Native code • Converges with Windows 8 • Improved one-shot location acquisition • <Map> XAML Control

6. New Windows Phone Runtime Location API Windows Phone 7 .NET location API FULLY SUPPORTED HOWEVER

7. Location Sources GPS + Accuracy - Power - Speed - Indoors - Accuracy + Power + Speed - Wilderness +/- Accuracy +/- Power +/- Speed +/- Urban areas WiFi Location Services Cell Towers

8. How to Get the Phone’s Current Location Geolocatorgeolocator = newGeolocator(); geolocator.DesiredAccuracyInMeters = 50; GeopositionmyLocation = awaitgeolocator.GetGeopositionAsync(maximumAge, timeout); // We need to convert the Geoposition to a GeoCoordinate to show it on the map GeoCoordinategeoCord = newGeoCoordinate(myLocation.Coordinate.Latitude, myLocation.Coordinate.Longitude); MyMapControl.Center = geoCord; // Set up an event handler to watch for location updates GeoCoordinategeolocator.PositionChanged += updatePosition; (Visual Studio)

9. Position Changed voidnewPosition(Geolocator sender, PositionChangedEventArgsargs) { Deployment.Current.Dispatcher.BeginInvoke(() => { GeoCoordinategeoCord = newGeoCoordinate(args.Position.Coordinate.Latitude, args.Position.Coordinate.Longitude); MyMapControl.Center = geoCord; }); }

10. Maps APIs on Windows Phone 8 • Windows Phone 8 has new Map control namespaces: • Microsoft.Phone.Maps • Microsoft.Phone.Maps.Controls • Microsoft.Phone.Maps.Services

11. Map Control • Add a Map to your UI <Grid x:Name="ContentPanel"> <maps:Map x:Name="MyMap"/></Grid> • In Code: privatevoidCreateMap() {MapMyMap = newMap();ContentPanel.Children.Add(MyMap);}

12. Built-In Map Tasks MapTask launch the map in 2 lines of code MapsDirectionTask start turn-by-turn directions in 4 lines of code MapsDownloaderTask download map data for offline in 2 lines of code

13. MapsDownloaderTask and Offline Maps MapDownloaderTaskmdt = newMapDownloaderTask(); mdt.Show(); Mapping offline in native and apps Route Queries do not require a network

14. Windows Phone 8 Sensors Maps Demo Location Adding UI Routing Offline

15. Background Location Tracking • The code shown in this sample gets a new location whenever the Periodic agent runs • Every 30 minutes or so • Windows Phone 8 supports continuous background location tracking • Suitable for Run Tracking apps and Turn-by-Turn navigation

16. Windows Phone 8 Sensors

17. Near Field Communications • Near Field Communications provide a connection between devices that are very close together (within 3-4 centimetres) • Transfer of a message or byte stream from one device to another • An NFC connection can be used to configure a connection which is implemented using Bluetooth or WiFi

18. Near Field Communications Options App to device App to app Tag to App

19. App to App communication • App to App communication allows two programs to interact using Bluetooth to exchange messages • The devices do not need to be paired in order to implement app to app connection • The ID_CAP_PROXIMITY capability must be enabled for the application

20. Publishing Messages using NFC ProximityDevice device = ProximityDevice.GetDefault(); // Make sure NFC is supported if (device != null) { _device.DeviceArrived += device_DeviceArrived; _device.SubscribeForMessage(“StringMessage", StringMessageHandler); _device.SubscribeForMessage(“ByteMessage", ByteMessageHandler); PeerFinder.Start(); }

21. Send Message When Device Arrives voiddevice_DeviceArrived(ProximityDevice sender) { if (sendingString) { sender.PublishMessage("MyStringMessage", someString); } elseif (sendingBytes) { varmessageWriter = newDataWriter(); messageWriter.UnicodeEncoding = UnicodeEncoding.Utf16LE; messageWriter.WriteBytes(someBytes); sender.PublishBinaryMessage("MyBinaryMessage", messageWriter.DetachBuffer()); } }

22. Receive String Message privatevoidStringMessageHandler(ProximityDevicesender, ProximityMessage message) { stringmessageRecieved = message.DataAsString; }

23. Receive Byte Message privatevoidByteMessageHandler(ProximityDevicesender, ProximityMessage message) { byte[] messageBytes; using(DataReaderdReader = DataReader.FromBuffer(message.Data)) { messageBytes= newbyte[dReader.UnconsumedBufferLength]; dReader.ReadBytes(messageBytes); } }

24. NFC socket connection to Bluetooth Socket

25. Setting up a StreamSocket using NFC ProximityDevice device = ProximityDevice.GetDefault(); // Make sure NFC is supported if (device != null) { PeerFinder.TriggeredConnectionStateChanged += OnTriggeredConnectionStateChanged; // Start finding peer apps, while making app discoverable by peers PeerFinder.Start(); } • The PeerFinder class allows an application to bind to an event fired when another application attempts to set up connection with this device

26. Setting up a StreamSocket using NFC voidOnTriggeredConnectionStateChanged(object sender, TriggeredConnectionStateChangedEventArgsargs) { switch (args.State) { caseTriggeredConnectState.Listening: // Connecting as host break; caseTriggeredConnectState.PeerFound: // Proximity gesture is complete – setting up link break; caseTriggeredConnectState.Connecting: // Connecting as a client break; caseTriggeredConnectState.Completed: // Connection completed, get the socket streamSocket= args.Socket; break; caseTriggeredConnectState.Canceled: // ongoing connection canceled break; caseTriggeredConnectState.Failed: // Connection was unsuccessful break; } } • The event arguments contain a state change message

27. Using the connection PeerFinder.AllowBluetooth = true; PeerFinder.AllowInfrastructure = true; • The StreamSocket will be created using WiFi or Bluetooth to transfer the data • An application can configure the networking technologies by setting these properties • They are both set to true by default

28. Blutooth Options There are two Bluetooth communication scenarios supported by Windows Phone App to app App to device No pairing required Pairing required

29. Interacting with Remote Devices via Bluetooth • The Bluetooth discovery and connection methods provide the raw ability to transfer data between the devices using the StreamSocket • The application will have to implement the communications protocol that is required for a particular device

30. Windows Phone 8 Sensors

31. Windows Phone 8 Sensors 15 supported languages Multiples voices per language

32. Windows Phone 8 Sensors Speech Synthesis (Text to Speech) Voice Commands Speech to text

33. Speech Synthesis

34. Simple Speech asyncvoidCheeseLiker() { SpeechSynthesizer synth = newSpeechSynthesizer(); awaitsynth.SpeakTextAsync("I like cheese."); } • The SpeechSynthesizer class provides a simple way to produce speech • The SpeakTextAsync method speaks the content of the string using the default voice

35. Speech Synthesis And Language Support • Set SpeechSynthesizer to different language foreach (VoiceInformation vi inInstalledVoices.All) { if (vi.Language == "de-DE") { _speechSynth= newSpeechSynthesizer(); _speechSynth.SetVoice(vi); } }

36. Voice Commands

37. Application Launching using Voice command • The Voice Command feature of Windows Phone 7 allowed users to start applications • In Windows Phone 8 the feature has been expanded to allow the user to request data from the application in the start command • The data will allow a particular application page to be selected when the program starts and can also pass request information to that page

38. Voice Command Query “Fortune Teller, Will I find gold?” Phone displays “Showing gold” /Money.xaml/?voiceCommandName=showMoney&futureMoney=gold&reco=Fortune%20Teller%Will%20I%20find%20gold"

39. Voice Command Query • /Money.xaml/ • ?voiceCommandName=showMoney • &futureMoney=gold • &reco=Fortune%20Teller%Will%20I%20find%20gold Phaselist Name Command Name Recognized phrase Whole phrase as it was recognized

40. Simple Speech Input

41. Recognizing Free Speech • A Windows Phone application can recognise words and phrases and pass them to your program • The standard “Listening” interface is displayed over your application

42. Simple Speech Recognition SpeechRecognizerUIrecoWithUI; asyncprivatevoidListenButton_Click(object sender, RoutedEventArgs e) { this.recoWithUI= newSpeechRecognizerUI(); SpeechRecognitionUIResultrecoResult = awaitrecoWithUI.RecognizeWithUIAsync(); if( recoResult.ResultStatus == SpeechRecognitionUIStatus.Succeeded) MessageBox.Show(string.Format("You said {0}.", recoResult.RecognitionResult.Text)); } • The above method checks for a successful response • By default the system uses the language settings on the Phone

43. Speech Recognition And Language Support • Set SpeechRecognizerUI to different language foreach(SpeechRecognizerInformationsri inInstalledSpeechRecognizers.All) { if(sri.Language == "de-DE") _speechRecognizer.Recognizer.SetRecognizer(sri); }

44. Windows Phone 8 Sensors Matthias Shapiro mashapir@microsoft.com http://matthiasshapiro.com