1 / 28

Wagner Silveira

Wagner Silveira. Microsoft MVP - Azure. Creating a Processing Pipeline with Azure Function and AIS. About me. wsilveiranz. wagner.silveira@notetoself.tech. https://notetoself.tech. Principal Integration Architect Microsoft Azure MVP MCSE Cloud Platform. Agenda. Case Study

sissy
Download Presentation

Wagner Silveira

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Wagner Silveira • Microsoft MVP - Azure Creating a Processing Pipeline with Azure Function and AIS

  2. About me wsilveiranz wagner.silveira@notetoself.tech https://notetoself.tech Principal Integration Architect Microsoft Azure MVP MCSE Cloud Platform

  3. Agenda • Case Study • Initial Solution • Updated Solution • Key Components • A year later – what changed? • Lessons Learned

  4. Case Study

  5. The solution requirements • Onboarding of external EDI messages • Should allow for different transports • Message need to be Validated, translated, enriched, routed • Control on retries and notification over failed messages • Auditing of initial message • Visibility of where a single message was in the process

  6. Logical Design

  7. Initial Solution

  8. Reality Check • Big Payloads • EDIFACT schema not available • Operational costs • End to end Monitoring

  9. Updated Solution

  10. How was this different from the initial solution? • Azure Functions • EDIFACT support via .NET Package • Claim check pattern • Dedicated instances • Azure Storage • Payload storage (claim check pattern) • Application Insights • End to End with custom events • Single technology for monitoring and notification

  11. Key Components

  12. Key Components • Claim check pattern • App Insights Custom Events

  13. The claim check pattern • Publishing function separates content from metadata then: • Store content on a safe repository • Publish metadata + storage pointer + Reprocessing count to message engine • Subscribing processes receive metadata and retrieve content when required

  14. App Insights custom events • End to end traceability using a Correlation ID • Allow for exceptions to be captured the same way • Leverage Functions support for App Insights • Implemented as a reusable component App Insights

  15. Hands on Time?

  16. Key Management • Leveraged MSI support where available • SQL Server • Storage • Key vault • Kept keys in key vault where MSI where not supported directly • Service Bus

  17. Exception Management & Retrieson Functions • Catch blocks using a notify and throw pattern • Leverage Function SB binding peek-lock pattern • Adjusted Maximum Delivery Count on SB • Messages that failed where delivered to DLQ

  18. DLQ Management • Logic Apps polling subscriptions DLQ every 6 hours • Each subscription DLQ could have it’s own logic • Reprocessing count • Resubmission logic • Notification logic • Email notification • Error blob storage

  19. One year later…

  20. New technology candidates • Integration Services Engine • Azure Durable Functions • Event Grid

  21. What about new features • Azure Functions Premium • Improved support for Key Vault • Improved support for MSI • Virtual Network support + Service Endpoints

  22. Lessons learned

  23. Review the fine print

  24. Operational cost is a design consideration

  25. Make the best of each technology

  26. Think about the big picture

  27. In Summary • Weight your technology options • Think about limitations, operational costs • Understand how your solution fit in the big picture • Understand the components roadmap

  28. Thank you for attending!

More Related