Supporting Akamai Products and Services

1. Supporting Akamai Products and Services David Reisfeld Manager Technical Consulting May 11, 2004

2. Agenda • Customer Care Overview • Technical Overview of EdgeSuite (20 min) • Customer Self-help Support Tools in EdgeControl (20 min) • Troubleshooting: How to identify a non-Akamai vs. Akamai Content Delivery issue (40 min) • Advanced EdgeDiagnostics Tools (40 min) • Live Troubleshooting Demo (20 min)

3. Customer Care Overview • Customer Care provides ongoing post-sales/post-integration support for technical issues • Track the issue and communicate the status and resolution to you-- serve as your point of contact with other departments within Akamai

4. Ticket profile • 3 types of issues • Break – fix • Can’t get content • Wrong content • Provisioning • EdgeSuite configuration change requests • Managing Akamai services • Informational requests • Questions about Akamai products • How do I … • Low volume calls • Attests to success of the products

5. Typical issues handled • Unavailable web pages or missing images • Slow performance on website or stream • Stale content appearing • How do I… • Upload content to Net Storage? • Create a new live stream? • Create a new login or reset a password for the EdgeControl Management Center? • Understand my traffic reports?

6. Customer Care facts… • 24x7 live availability • Offices in Cambridge, San Mateo, Tokyo, Munich, Paris • Phone line open 24x7 • E-mail monitored during business hours: • Business hours is 9 a.m. to 9 p.m. ET • 24x7 on-line issue tracking and tools • https://control.akamai.com • SLA for response times • Routine or High Priority or Urgent

7. Engaging AkamaiRoles and Responsibilities • Reseller Support: • Receive the first call for customer technical questions • Independently resolve all non-Akamai-related issues • Use the EdgeControl Management Center http://control.akamai.com to manage and modify Akamai services • Collect key problem information before escalation • Perform EdgeDiagnostics if necessary • Escalate to Akamai via online Issue Tracking with appropriate information • Akamai CCare: • Provide Tier 2+ support for all Akamai-specific system & network issues • Maintain and update portal resources incorporating customer feedback • Uphold SLA response times to escalations from Reseller

8. Engaging AkamaiPriority Levels and Service Level Agreement (SLA)

9. Agenda • Customer Care Overview (20 min) • Technical Overview of EdgeSuite (20 min) • Customer Self-help Support Tools in EdgeControl (20 min) • Troubleshooting: How to identify a non-Akamai vs. Akamai Content Delivery issue (40 min) • Advanced EdgeDiagnostics Tools (40 min) • Live Troubleshooting Demo (20 min)

10. Some terminology

11. Ordinary HTTP content delivery 2. DNS lookup www.foo.com? 1. DNS lookup www.foo.com? 3. DNS reply 11.22.33.44 4. DNS reply 11.22.33.44 Client DNS Server Origin DNS Authority End User Client Machine 5. GET / HTTP/1.1 The Internet 6. HTTP Response Origin Server 11.22.33.44

12. How FreeFlow/EdgeSuite Object Delivery Works 1. DNS lookup a4.g.akamai.net? Client DNS Server Origin DNS Authority 2. DNS reply Akamai server 4. DNS lookup www.foo.com? 5. DNS reply 11.22.33.44 3. GET /f/2/3/10m/www.foo.com/ HTTP/1.1 6. GET / HTTP/1.1 8. HTTP Response End User 7. HTTP Response Akamai Edge Servers Origin Server 11.22.33.44 Note: HTML pages are delivered from the origin server

13. How EdgeSuite Whole Site Delivery Works 1. DNS lookup www.foo.com? Client DNS Server Origin DNS Authority 2. DNS reply CNAME to Akamai 4. DNS lookup origin.foo.com? 3. GET / HTTP/1.1 5. DNS reply 11.22.33.44 8. HTTP Response 6. GET / HTTP/1.1 End User 7. HTTP Response Akamai Edge Servers Origin Server Note: HTML pages and objects are delivered from the AKAMAI network

14. EdgeSuite Configuration • Deployed custom EdgeSuite configuration files (metadata) for every customer • Access control list for all origin server domains and IPs with their valid CP codes

15. Sample Configuration Metadata <originMap tree="2 content.foo.com"> <originServer value="origin.content.foo.com"> <hostHeader>content.foo.com</hostHeader> <hostHeader>test.content.foo.com</hostHeader> <hostHeader>content.foo.com.edgesuite.net</hostHeader> </originServer> </originMap> … <tree name="2 content.foo.com"> <md name="use-hierarchy">coremap</md> <md name="max-age">1h</md> <md name="cpcode">1234</md> <md name="serial">972</md> <md name="enable-post">on</md> <md name="log-referrer">on</md> <md name="log-hosthdr">on</md> <md name="log-cookies">on</md> <md name="log-user-agent">on</md> <match type="arltype" value="transparent"> <md name="fwd-hostheader">content.foo.com</md> </match> <match type="ext" value="aif aiff au avi bin bmp cab cdf class css dcr exe gif hqx jpg jpeg js mov mp3 nc pct pdf pws swf txt wav zip"> <md name="max-age">1d</md> </match> </tree>

16. EdgeSuite Variation – Net Storage 1. DNS lookup www.foo.com? Client DNS Server Origin DNS Authority 2. DNS reply CNAME to Akamai 4. DNS lookup www.foo.com? 3. GET / HTTP/1.1 5. DNS reply 11.22.33.44 8. HTTP Response 6. GET / HTTP/1.1 End User 7. HTTP Response Akamai Edge Servers Akamai Net Storage. Fault tolerant replicated servers on a custom domain: foo.download.akamai.com

17. EdgeSuite Variation – Tiered Distribution Origin DNS Authority End User DNS lookup www.foo.com? DNS reply 11.22.33.44 Akamai edge server contacts Tiered Distribution Parent server GET / HTTP/1.1 HTTP Response Origin Server Akamai Edge Servers

18. Agenda • Customer Care Overview (20 min) • Technical Overview of EdgeSuite (20 min) • Customer Self-help Support Tools in EdgeControl (20 min) • Troubleshooting: How to identify a non-Akamai vs. Akamai Content Delivery issue (40 min) • Advanced EdgeDiagnostics Tools (40 min) • Live Troubleshooting Demo (20 min)

19. The Manage Tab • Configure Alerts • Refresh content in Content Control Utility • Maintain the Domain Validation List • Provision Net Storage • Provision Live Streams

20. Alerts • Provide automatic, real-time notification via email • Upon occurrence of condition • When condition clears • Monitor origin infrastructure, detect changes in: • Performance • Traffic volume Customers manage their own Alerts on the EdgeControl Management Center

21. Some Alert Types • High Traffic • Traffic above threshold (e.g. flash crowd) • Low Traffic • Traffic below threshold (e.g. network outage, origin server problem) • Origin Connection Failure • Connections to origin servers fail • Origin DNS Failure • DNS requests resolving to origin server domain names fail • Origin Server Failure • Requests to origin fail for unknown reasons • Origin Object Not Found • Requests to origin server fail because object not found • Origin Access Denied • Requests fail because access to origin denied

22. Content Control Utility • The Content Control utility is a web-based tool which allows customers to manually refresh content by URL on the Akamai network without waiting for the TTL to expire • Customers can • purge: evict content from all caches completely • invalidate: mark all currently cached copies as stale and thus triggering only an If-Modified-Since request to the origin

23. Enhanced Content Control Utility • The Enhanced Content Control Utility (ECCU) is a web-based tool which allows specifying RULES for cached objects to refresh • Allows you to refresh • All content with particular extension (gif, jpg, etc.) • All content under a particular path (i.e., /abc/images.) • All content with a particular extension and path • You may also purge by uploading a ECCU metadata file (see documentation) • The Enhanced CCU uses an invalidation-based refresh method

24. Content Control Utility API • The Akamai Content Control Utility SOAP API (CCUAPI) provides a mechanism for Akamai customers to write programs that submit a list of cached objects to be removed from the Akamai network. CCUAPI uses Simple Object Access Protocol (SOAP) and HTTP or HTTPS as a transport layer. • The CCUAPI uses either the removal-based or the invalidation-based refresh method • Available in: Perl Java Visual Basic

25. Domain Validation Tool ! All origin server domains or IP addresses must be registered with Akamai for the edge server access control list. Unregistered domains will result in a 403 Forbidden denial of service. Customer manage their own Valid Domains in the EdgeControl Management Center.

26. Net Storage • Akamai Net Storage is a managed service that provides persistent, replicated storage of website content • Can store images, html, streaming media files, software, documents, and other digital objects. • Can serve as content origin, thus offloading origin bandwidth usage, or as a failover website • Can upload via FTP, rsync, scp, or shell-prompt • Can upload via regular or secure (ssh) connections • Customers manage their own Net Storage accounts and logins in the EdgeControl Management Center

27. Akamai Live Streams • Akamai supports WMS, Real, and Quicktime • Customer can provision and modify their own streams in the EdgeControl Management Center. • Use the EdgeControl Management Center to find information about any stream

28. Other ECMC gems – Monitor and Reports Tabs • Monitor - View live hits and bits for any customer in real time • Event Viewer - View records of your customers modifying and managing their services • Reports – View historical hits and bits for any customer for up to 3 months

29. Other ECMC gems - EdgeSuite Booster • EdgeSuite Booster Version 1.3.2 (Windows Installer executable) • The EdgeSuite Booster is a tool to allow users to see how their EdgeSuited pages are cached. It adds menu entries to Internet Explorer's context menu. Also included is the Akamai Hosts Toggle program, which allows users to send their requests to the BETN. This install package will install Microsoft XML parser 4.0 if it is not already installed. • Link located under Support -> Documentation -> EdgeSuite

30. Agenda • Customer Care Overview (20 min) • Technical Overview of EdgeSuite (20 min) • Customer Self-help Support Tools in EdgeControl (20 min) • Troubleshooting: How to identify a non-Akamai vs. Akamai Content Delivery issue (40 min) • Advanced EdgeDiagnostics Tools (40 min) • Live Troubleshooting Demo (20 min)

31. Basic Information about a Problem • SCOPE • Who is experiencing the issue, one or many users? • Can you reproduce the problem locally? • SERVER • Is Akamai serving the content? • Which Akamai server? • CONNECTIVITY • Does the user have connectivity to the server? • FREQUENCY • When did problem start? • Is it occurring now?

32. SCOPE Akamai is a globally distributed network. Always attempt to reproduce the problem locally!

33. SERVER

34. CONNECTIVITY: Successful content delivery transactions require connections…

35. FREQUENCY

36. 6 Ingredients for successful delivery • The Client’s DNS resolution to EdgeSuite CNAME and edge server IP address • The Client’s network connectivity to the resulting Akamai server IP in both directions Demo on: content.foo.com • The response from edge server IP address for the HTTP request • The edge server’s DNS resolution for the origin server DNS name • The edge server’s network connectivity to the origin server datacenter in both directions • The origin server’s response for the HTTP request

37. Recommended publicly available tools • DNS • nslookup, dig • Network Information • mtr, winmtr, tracert, traceroute, ping, netstat • General download simulation • wfetch, curl, various browsers • Header and Akamai specific info • EdgeSuite Booster: tells you which Akamai server you got • Local DNS Hosts file alteration • C:\WINNT\System32\drivers\etc\hosts • Mozilla Firefox browser with Live HTTP headers plug-in • TCP packet information (rare) • Ethereal, PacketBoy, tcpdump

38. Akamai on-line documentation • http://control.akamai.com • Support -> Documentation -> reference by product name • Context-sensitive help in the “?” icons for most of the on-line applications • Ask Akamai, a searchable on-line knowledge base maintained by Akamai Customer Care

39. Akamai’s Knowledge Base

40. Agenda • Customer Care Overview (20 min) • Technical Overview of EdgeSuite (20 min) • Customer Self-help Support Tools in EdgeControl (20 min) • Troubleshooting: How to identify a non-Akamai vs. Akamai Content Delivery issue (40 min) • Advanced EdgeDiagnostics Tools (40 min) • Live Troubleshooting Demo (20 min)

41. Locating Edge Diagnostics • The EdgeControl Management Center • https://control.akamai.com/

42. Available Edge Diagnostics • Get End User IP Information • Gets basic IP information for an end user. • Translate an ARL • Gets basic info (TTL, origin server) for an Akamaized URL • Get Domain Information (nslookup/dig) • Resolves DNS hostnames from an Akamai edge server • Request Content from a Region or Origin (curl) • Fetches the HTTP response headers from a request • View Logs (grep) • Shows HTTP response codes, time-taken, bytes delivered, and end-user IP addresses of requests from Akamai server logs • Network Connectivity Test (mtr) • Checks network conditions between an Akamai server and an Internet host, client or origin server

43. Scenario #1: Broken Image • In this scenario, a user reports a broken image on the Web site • What to do? • Get basic URL information • Check if image exists on origin server • Check if problem is reproducible at Akamai server • Check if Akamai server can fetch content from origin • Check logs on the Akamai server • How to troubleshoot? • Follow steps in subsequent slides…

44. Get Basic URL information • Use Akamai Translate to see basic URL info • Note the origin server and the CP code

45. Get Basic URL Information (Results)

46. Check if image exists on origin server • Replace the EdgeSuite CNAME domain with the origin server domain, download object locally • If multiple origin servers exist, check each

47. Check if image exists on origin server (Results)

48. Check if problem is reproducible at Akamai server • Ask end user to run nslookup to the EdgeSuite CNAME domain to get Akamai IP • Spoof DNS Hosts file • C:\WINNT\system32\drivers\etc\hosts • <IP address> <Edgesuite domain> • Try to download object from specific Akamai IP(s) using a browser

49. Check if problem is reproducible at Akamai server (Results)

50. Check if Akamai server can fetch content from origin • Use Request Content to see HTTP headers • Check for correct object size, status code, etc.