Zero to Hero with Microsoft Graph Real-world lessons from 1B+ api calls

2. Zero to Hero with Microsoft GraphReal-world lessons from 1B+ api calls Chris Johnson, CTO – Hyperfish@c_f_johnson

3. But first …When at first you don’t succeed.

4. Why am I up here? • 2+ years building products with graph at Hyperfish • Made lots of all the mistakes along the way • Many lessons learnt • I hope you learn something

5. The Graph and Hyperfish • Analysis Profiles • Users, Photos, Manager, Delta Query, App-Only auth • Access Profiles • Users, Photos, Manager, Delta Query • Profile Updates • Users, Photo, Manager • Email – send • Send Mail

6. Why did we choose Graph? “One throat to choke” One throat to choke is an expression used in business to describe the advantage of purchasing goods or integrated services from a single vendor. That way, when something goes wrong, there is only “one throat to choke.” “Modern” Single API surface across Microsoft All types of apps

7. Hype or the cat's pajamas?

8. Build 2015 REST/JSON Production use cases “it’s working!” #fail - API X isn’t in graph

9. Getting started

10. Auth that’s easy – learn to auth in 5 mins • Create App ID/Secret • Authorize /authorize • Get a token /token • Profit /users

11. Auth – the final frontier • v1 vs. v2 app registrations • Dynamic consent • Personal vs. Work accounts • Registration portal (AAD vs. apps.dev.microsoft.com) • ADAL vs. MSAL • Access Tokens • They are opaque • You can’t validate them • http://www.cloudidentity.com/blog/2018/04/20/clients-shouldnt-peek-inside-access-tokens/ • Refresh Tokens • Don’t try & figure out when they expire 

12. Delegated or “App-Only”? Delegated On-behalf of a user Access Token + Refresh Token Access to things the user has access to “App-Only” / Client Credential Flow App ID and Secret Admin consent only Get Access Tokens as often as you like No expiry Good for user not present scenarios

13. Dynamic Consent v2 apps only Ask for more as you need it Not available for “app-only” Use for Feature X that is optional and needs more

14. Auth the hard way in 5 mins (hopefully)

15. My Top 5 lessons, tips & tricks for awesome times with MS Graph

16. Tip #1: Build with performance in mind

17. Our experience • Get page of X users • For each user: • /photo • /manager • If any of the calls fail try 3 times before giving up • e.g. 100k users • 900 /page • 111 page requests • 100k photos • 100k managers • 200,111 requests – best case • 600,333 worst case

18. Multiple requests Single request

19. Using batch Get page of X users Get batch of X /photos + /manager 8x to 10x improvement Latency improvements Concurrency improvements e.g. 100k users 111 page requests 10k batch requests 10,111 requests in total ~20x improvement ~8x to 10x in practice

20. Demo

21. Tip #2: Get changes, don’t request it all again.

22. Our experience • Before • Analyzing profiles for org • Page through entire org • Do related queries • Fine for ~2k – 3k users • Not fine for 500k user orgs

23. Delta query (aka track changes) deltaLink Gives you an opaque bookmark Original query is preserved $select/$filter to indicate what you want to track Entities Users, OneDrive, Contacts, Messages, Mail, Groups, Calendar, Planner

24. Demo

25. Tip #3: Don’t poll if you can help it

26. Our experience Scheduled analysis Once a day full analysis 5 min incremental analysis Poll users for changes Analyze etc… Pros: Simple Cons: timers are painful, overhead for nothing

27. Webhooks Webhooks + Delta Query = awesome No more polling! Web hooks are fragile … because internet Delta query is your “last seen” bookmark Example: Query /users/delta, page through and store the deltaLink Subscribe to for a web hook e.g. /users, changed Receive a webhook Use deltaLink to query for all changes since last time

28. Demo

29. Tip #4: Back-off punk

30. Throttling What you need to handle HTTP 429 429 TOO MANY REQUESTS What you need to do about it HTTP Header Retry-After: 3600

31. DoS level throttling .. oops ECONNRESET Connection refused

32. Demo

33. Tip #5: When the graph doesn’t have what you need

34. Calling other APIs • “multi-resource refresh tokens” • e.g. SharePoint Online • Scope: https://<tenant>.sharepoint.com/Sites.Read.All • e.g. Azure AD graph • Scope: https://graph.windows.net/.default

35. Demo

36. Bonus round “null” it doesn’t mean null might be you don’t have permissions. You have no way to tell ¯\_(ツ)_/¯ When things go wrong keep: Your query Response: request-id and date { "error": { "code": "Authorization_RequestDenied", "message": "Insufficient privileges to complete the operation.", "innerError": { "request-id": "b6cbbbe6-f038-4ed5-89e5-5426f301b091", "date": "2018-05-17T19:58:04" } } }

37. Where to go https://stackoverflow.com/questions/tagged/microsoft-graph - Questions - Feature Requests https://officespdev.uservoice.com/ - Documentation issues/requests https://github.com/microsoftgraph/microsoft-graph-docs

38. When you found a bug or production issue

39. So what now ? graph.microsoft.com Postman Graph Explorer Have fun! Batch Delta Webooks Throttling Other endpoints

40. Booth #205

41. Microsoft Cloud Show Weekly show News, Interviews and topic based Microsoft focused @mscloudshow microsoftcloudshow.com

42. Get in touch… @c_f_johnson www.chrisjohnson.io www.hyperfish.com cj@hyperfish.com

43. Q&A