1 / 48

“ ”

Overview of building IoT solutions. “ ”. Sergii Bielskyi. About me. Current – Cloud architect at Eleks Previous – Cloud Solution Architect at Qiio E-doc at APU MCS at MS Ukraine MVP (5 times), Lead of SharePoint and IoT communities

davisc
Download Presentation

“ ”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of building IoT solutions “ ” SergiiBielskyi

  2. About me Current – Cloud architect at Eleks Previous – Cloud Solution Architect at Qiio E-doc at APU MCS at MS Ukraine MVP (5 times), Lead of SharePoint and IoT communities Facebookhttps://www.facebook.com/sergey.belskiy Twitter@sergey_belskiy Bloghttps://medium.com/@sergiibielskyi IoT communityhttps://www.facebook.com/groups/iot.ua/

  3. What is IoT, and why is it important? Agenda IoT from an Architect’s perspective Technologies Architects need to know Common patterns and practices

  4. 25 Billion (2020) Source: Gartner

  5. what is it? Unique objects connected to Internet Devices, not people Bi-directional communication Large, complex data flows New types of insight

  6. IoT 2010 IoT 2010 Alarm clock Refrigerator Automobile VoIP phone Computer Printer Media player Microwave HOME WORKPLACE HOME Coffee maker Smart scale Oven Cell phone HVAC Vending Security Television

  7. IoT 2019 IoT 2015 Pet tracking Medication adherence Object tracking Behavior modification Health monitoring Sports and fitness Child and eldermonitoring Beacons and proximity Smart lighting Indoor navigation Trip tracking and car health HOME WORKPLACE HOME Smart vending machines Office equipment Identity Bike ride stats and protection Smart appliances Air conditioning and temperature control Environmental sensors Control Information capture Sleep tracking Entertainment systems Food and nutrition tracking Garden, lawn and plant care New devices and sensors Leak detection Home security Home automation

  8. IoT architecture requirements Handle extreme hardware and software heterogeneity. Build for hyper-scale and enable low data latency. Be secure by design; support defense in depth. Lower barriers to entry: evaluate -> prototype -> deploy. Deliver telemetry and notifications that are meaningful even at extreme scale. Provide hot-path and cold-path analysis and action/response.

  9. Information Exchange PatternsBasics of IoT Communication Telemetry Inquiries Commands Notifications Information flowing from a device to other systems for conveying status of device and environment Requests from devices looking to gather required information or to initiate activities Commands from other systems to a device or a group of devices to perform specific activities Information flowing from other systems to a device (group) for conveying status changes

  10. Azure IoT Reference Architecture Device and Event Processing Devices and Data Sources Data Transport Presentation Data Visualization & Presentation Cloud Gate-way Provisioning API Solution Portal IP capable devices Device State Store Identity & Registry Stores AgentLibs Existing IoT devices Stream Event Processor Gateway AgentLibs Storage Analytics/ Machine Learning AgentLibs Low power devices Control System Worker Role

  11. What Do Architects Need to Know to build the solution?

  12. Microsoft Azure IoT Services { }

  13. Event Hubs Telemetry Ingestion Services • Scalable publish-subscribe telemetry ingestors • Processes massive amounts of data (Generally Available WW) • 1M Publishers • 1 GB/S Ingress • 1T Messages/Month (1k per message) • TTP/AMQP Protocol Support • Pluggable adapters for other cloud services

  14. CosmosDB Fully managed NoSQL document db service { } NoSQL JSON and JavaScript DB Service JSON DB for rapid Development Schema free – for storage and query Automatic indexing of every document property CRUD access, query, and JavaScript processing Integrates with HDInsight, Azure Search, etc. { }

  15. Azure HDInsight 100% Apache Hadoop-based service in the cloud Scales from terabytes to petabytes on demand Processes unstructured or semi-structured data from devices and sensors Deployable in Windows or Linux Connects with on-premises Hadoop clusters Apache Storm for real-time events Apache Spark for in memory data analysis

  16. Azure Stream Analytics Process real-time data in Azure. “Data in Motion” Real Time Analytics Intake millions of events per second (up to 1 GB/s) Correlate between different streams, or with static data or models Easy processing on continuous streams of data Enables the detection of anomalies. Ability to trigger an alert when a specific error or condition appears.

  17. Azure Machine Learning Cloud based predictive analytics engine Designed for “applied” machine learning, Streamlined experience for data scientists, across multiple skill levels Drag-and-drop, and data-flow graphs to set up experiments Build and test predictive models, predict future trends or behavior Publish models as a fully managed web service (API) Transform and Analyze Publish Data Sources Ingest Raw materials Acquire Raw Materials Transform raw materials into “finished goods” Deliver

  18. Azure Notification Hubs Scalable mobile push notification engine Target millions of devices/messages with single API call. Target audiences with dynamic tags. Tailor notifications by audience, language, and location Use with any back end, in the cloud or on-premises Dynamically define and reach audience segments

  19. Microsoft PowerBI Live, single pane of glass dashboard solution for visualizations and KPIs • Cloud based business analytics service: • Track data in real-time with support for streaming data • Drill through to underlying reports to explore and discover new insight • Pin new visualizations and KPIs to monitor performance

  20. IoT Cloud Patterns

  21. IoT Device & Cloud Patterns Cloud Gateway Event Hubs • Event Hubs • High scale telemetry ingestion service • HTTP/AMQP protocol support • Each Event Hub supports • 1 million publishers • 1GB/s ingress • Generally available worldwide • 1 Trillion messages/month Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Field Gateway Field Gateway Device Connectivity & Management

  22. IoT Device & Cloud Patterns Cloud Gateway Event Hubs • Additional IoT Requirements • Command & control • Device identity • Device registry • Device management Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Field Gateway Field Gateway Device Connectivity & Management

  23. IoT Device & Cloud Patterns Cloud Gateway Event Hubs & IOT Hub • IoT Hub • Capability coming with Azure IoT Suite • Bi directional D2C and C2D • Up to 10M devices • Telemetry ingestion • Command & control • Device registry & identity • Device Management • More… Protocol Adaptation Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Field Gateway Field Gateway Device Connectivity & Management

  24. IoT Device & Cloud Patterns Batch “Cold Path” Analytics Azure HDInsight, DocumentDB, Azure ML Cloud Gateway Event Hubs & IOT Hub { } { } Hot Path Analytics Azure Stream Analytics, Azure HDInsight (Storm) Protocol Adaptation Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Hot Path Business Logic Service Fabric & Actor Framework Field Gateway Field Gateway Device Connectivity & Management Analytics & Operationalized Insights

  25. IoT Device & Cloud Patterns Batch “Cold Path” Analytics Azure HDInsight, DocumentDB, Azure ML Presentation & Business Connectivity App Service, Websites Dynamics, BizTalk Services, Notification Hubs Cloud Gateway Event Hubs & IOT Hub { } { } Hot Path Analytics Azure Stream Analytics, Azure HDInsight (Storm) Protocol Adaptation Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Hot Path Business Logic Service Fabric & Actor Framework Field Gateway Field Gateway Device Connectivity & Management Analytics & Operationalized Insights Presentation & Business Connectivity

  26. Pattern: Predictive Maintenance Batch “Cold Path” Analytics Azure HDInsight, DocumentDB, Azure ML Presentation & Business Connectivity App Service, Websites Dynamics, BizTalk Services, Notification Hubs Cloud Gateway Event Hubs & IOT Hub { } { } Hot Path Analytics Azure Stream Analytics, Azure HDInsight (Storm) Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Hot Path Business Logic Service Fabric & Actor Framework Device Connectivity & Management Analytics & Operationalized Insights Presentation & Business Connectivity

  27. Pattern: Service Delivery Management Batch “Cold Path” Analytics Azure HDInsight, DocumentDB, Azure ML Presentation & Business Connectivity App Service, Websites Dynamics, BizTalk Services, Notification Hubs Cloud Gateway Event Hubs & IOT Hub { } { } Hot Path Analytics Azure Stream Analytics, Azure HDInsight (Storm) Protocol Adaptation Devices RTOS, Linux, Windows, Android, iOS Hot Path Business Logic Service Fabric & Actor Framework Field Gateway Field Gateway Device Connectivity & Management Analytics & Operationalized Insights Presentation & Business Connectivity

  28. Examples

  29. Example Architecture Stream Analytics processes events as they arrive in the EventHub Event Hub Stores Streaming Data Data for Real-time Processing Power BI / D3 Dashboard Real time data stats Real Time Telemetry Data Aggregations Real Time Batch updates of predictions Batch Azure Data Factory Pipeline invokes AML Web Service AML Model Web Service BES endpoint Azure SQL Contains Historical Data Azure Data Factory Pipeline Moves Data Azure Services External Data

  30. Example Architecture Stream Analytics processes events as they arrive in the EventHub Azure WebJob Runs jobs to scrape data from public source Event Hub Stores Streaming Data Scrape Data 5 mins Data Stream Job Data for Real-time Processing Real Time Energy Consumption Data (Public Source) Real time data stats Hourly Prediction Updates Power BI / D3 Dashboard Real Time Batch Copy to Azure SQL for batch predictions Azure Data Factory Pipeline invokes AML Web Service AML Model Web Service BES endpoint Azure SQL Contains Historical Energy Consumption Data Azure Services External Data

  31. Example Architecture IoT Clients Azure Services Stream Analytics processes events as they arrive in the EventHub ML Predictions consumed through the RRS web service interface Data for Real-time Processing Streaming Data Real Time Telemetry Data Event Hub Stores Streaming Data AML Model (Published Web Service) Near Real-time Updates Power BI (Dashboard) IoT Clients Azure Services

  32. Preferred solution

  33. Existing accelerators

  34. Azure IoT Suite Azure IoT Suite IoT Hub Event processing and insight (e.g. Azure Stream Analytics) D2C receive endpoint IP-capable Cloud protocol gateway Device id C2D send endpoint Application device runtime logic Field gateway D2C send endpoint Msg feedback and monitoring endpoint PAN-devices Device … C2D queue endpoint Device … Application device provisioning and management Device identity management IoT Hub management Device … field cloud

  35. IoT Hub Many aspects of connectivity Cloud protocol gateways Functionality Device-to-cloud telemetry, Cloud-to-device commands and notifications, Bulk uploads/downloads Security Device security, Cloud security, Channel security, … Monitoring Identify malfunctioning devices when they cannot be reached directly Reach and customization RTOS/Linux/Windows/non-IP capable, Network/application protocols, Authentication schemes Field gateways Azure IoT Suite …

  36. Device provisioning Provisioned Making devices known to your system Many systems involved(IoT Hub, device registry, ERPs, …) Device identity(composite devices, many concerns) Sample provisioning Device provisioned at manufacturing into system Device connects for the first time and gets associated to its regional data center (bootstrapped) As a result of customer interactions the device is activated Devices can be deactivated for security and other reasons A device can also be de-provisioned at end-of-life or decommission. Bootstrapped Activated Deactivated De-provisioned

  37. Device-to-cloud messages Interface AMQP and HTTPS device-side endpoint AMQP service-side endpoint Device and service SDKs Compatible with Event Hubs Partitioned receiver, client check-pointing Integrations with Azure Stream Analytics, Storm, … IoT Hub services for D2C Millions of simultaneously connected devices Per-device authentication Connection-multiplexing: C2D and D2C traffic Across multiple devices for gateway scenarios Azure IoT Suite: IoT Hub D2C receive endpoint C2D send endpoint Device id D2C send endpoint C2D queue endpoint Device … Device … Device identity management Device …

  38. Cloud-to-device messages Interface AMQP and HTTPS device-side endpoint AMQP service-side endpoint At-least-once semantics Durable messages Device acknowledges receipt(Send - Receive - Abandon OR Complete) TTL and receipts Per-message TTL Per-message positive and negative receipts Command lifecycle pattern Use correlated D2C for responses Use feedback information to retry Store command state in command registry Azure IoT Suite: IoT Hub D2C receive endpoint C2D send endpoint Device id Msg feedback and monitoring endpoint D2C send endpoint C2D queue endpoint Device … Device … Device identity management Device …

  39. Azure IoT Suite SDKs Device-facing For devices and field gateway Platforms RTOS (FreeRTOS) Linux(Ubuntu, Debian, Fedora, Raspbian, Angstrom) Windows 7/8/10 ARM mbed Android iOS … Languages C, Java, C#, Javascript Service-facing For back-ends and cloud gateway Languages .NET C# Java Node

  40. C device SDK – D2C typedefstruct TELEMETRY_INSTANCE_TAG { DEVICEHUB_MESSAGE_HANDLE messageHandle; intmessageTrackingId; // For tracking the messages within the user callback. } TELEMETRY_INSTANCE; RcvConfirmationCallback(IOTHUB_CLIENT_HANDLE iotHubClientHandle, IOTHUB_CLIENT_CONFIRMATION_RESULT result, void* userContextCallback) {...} int main(void) { IOTHUB_CLIENT_CONFIG iotHubClientConfig; TELEMETRY_INSTANCE telemetryInstance; constchar* msgText = "Hello from IoTHubClient"; ... iotHubClientHandle = DeviceHubClient_Create(&deviceHubClientConfig); ... IotHubMessage_SetData(telemetryInstance.messageHandle, msgText, strlen(msgText)); ... IotHubClient_SendTelemetryAsync(deviceHubClientHandle, telemetryInstance.messageHandle, RcvConfirmationCallback, &telemetryInstance);... }

  41. C device SDK – C2D staticintReceiveNotificationCallback(IOTHUB_CLIENT_HANDLE iotHubClientHandle, IOTHUB_MESSAGE_HANDLE notificationMessage) { /* Some device specific action code goes here... */} int main(void) { IOTHUB_CLIENT_CONFIG iotHubClientConfig; IOTHUB_CLIENT_HANDLE iotHubClientHandle; iotHubClientConfig.iotHubName = “{name}"; iotHubClientConfig.deviceId = “{device id}"; iotHubClientConfig.deviceKey = “{key}"; iotHubClientConfig.protocol = IOTHUB_CLIENT_AMQP; iotHubClientHandle = IotHubClient_Create(&iotHubClientConfig) IotHubClient_SetNotificationCallback(iotHubClientHandle, ReceiveNotificationCallback) /* Wait for messages to arrive... */ }

  42. JavaScript (to Event Hub) vareventBody = { "reading": x, "device_id": id }; ehClient = new EventHubClient({ 'name': "kevinmil-demo", 'namespace': "kevinmil-demo-ns", 'sasKey': <snipped>, 'sasKeyName': "sendTelemetry", 'timeOut': 10, }); var msg = new EventData(eventBody); ehClient.sendMessage(msg, function (messagingResult) { // <body snipped> });

  43. Stream Analytics (to blob) SELECT device_id as Device_Id, reading as Reading, EventProcessedUtcTime as UTCDateTime FROM [eventhub] INTO [out2blob]

  44. Internet of Things DEVICES Summary OBJECTS CONSUMERS Think big (architecture), but start small (experiment, learn and refine). Start with telemetry. Address privacy, security and manageability before moving to command and control. Don’t interrupt the fast path and create processing bottlenecks. Think about security, identity and management from the very beginning, and through the life of the product. Build to the reference architecture to ease the move to IoT Suite. NETWORK BIG DATA STANDARDS INNOVATION BUSINESS SECURITY

  45. Stay tuned and learn more https://aka.ms/IoTShow https://aka.ms/IoTSchool

  46. Time to start https://microsoft.github.io/azure-iot-developer-kit/

  47. VS code extensions

  48. ThanksQ/A

More Related