REST Easy

1. REST Easy Peter Lacey Senior Consultant placey@burtongroup.com www.burtongroup.com Tuesday – June 26th, 2007

2. REST Easy • Agenda • Understanding REST • Using HTTP to build REST Applications • REST concepts and practices • Related technology • REST, WS-*, and SOA • Recommendations

3. REST Easy • Agenda • Understanding REST • Using HTTP to build REST Applications • REST concepts and practices • Related technology • REST, WS-*, and SOA • Recommendations

4. Understanding REST • What is REST? Part 1 “With REST, every piece of information has its own URL.” David Meggison, REST: the quick pitch Inventor of SAX

5. Understanding REST • What is REST? Part 2 • REST is an architectural style for distributed systems • An architectural style is… • … a coordinated set of architectural constraints that restricts the roles/features of architectural elements and the allowed relationships among those elements within any architecture that conforms to that style. — or — • … an abstraction, a design pattern, a way of discussing an architecture without concern for its implementation. • Music styles: Jazz, Blues (sorta) • Building styles: Baroque, Bauhaus (kinda) • Programming styles: Object oriented, functional, procedural

6. Understanding REST • What is REST? Part 2 • REST defines a series of constraints for distributed systems that together achieve the properties of: • A system that exhibits all defined constraints is RESTful • Systems may add additional constraints or relax existing constraints, in which case they are more or less RESTful • REST is not a set of laws or even a specification. It’s a style • Simplicity • Scalability • Modifiability • Performance • Visibility (to monitoring) • Portability • Reliability “I am a big fan of REST (evangelical even) but I will personally break every single principle of REST to get a job done.” Jon Hanna

7. Understanding REST • What is REST? Part 3: Representational State Transfer • Key information (data) abstraction: The Resource • A resource is any information that can be named: documents, images, services, people, collections, etc. • Resources have state • State may change over time • Resources have identifiers (constraint) • A resource is anything important enough to be referenced • Resources expose a uniform interface (constraint) • System architecture simplified, visibility improved. Encourages independent evolvability of implementations.

8. Understanding REST • What is REST? Part 3: Representational State Transfer • On request, a resource may transfer a representation of its state to a client • Necessitates a client-server architecture (constraint) • A client may transfer a proposed representation to a resource • Manipulation of resources through representations (constraint) • Representations returned from the server should link to additional application state. Clients may follow a proposed link and assume a new state • Hypermedia as the engine of application state (constraint)

9. Understanding REST • What is REST? Part 3: Representational State Transfer • Stateless interactions (constraint) • Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server • Statelessness necessitates self-descriptive messages (constraint) • Standard media types • Meta-data and control-data • Uniform interface + Stateless + Self-descriptive = Cacheable (constraint) • Cacheable necessitates a layered-system (constraint)

10. Understanding REST Summary of REST constraints

11. Understanding REST • Why bother? “There are irreconcilable differences between local and distributed computing” Jim Waldo et al, A Note on Distributed Computing • You cannot pretend remote objects are in the same address space as local objects. • The Eight Fallacies of Network Programming • 1. The network is reliable • 2. Latency is zero • 3. Bandwidth is infinite • 4. The network is secure • 5. Topology doesn't change • 6. There is one administrator • 7. Transport cost is zero • 8. The network is homogeneous

12. Understanding REST • Why bother? • Constrained systems emphasize restraint and awareness of the system’s context • Constraints are the realization of design goals • Unconstrained distributed systems are difficult to use, brittle, unreliable, and of marginal utility

13. Understanding REST • What is REST: the answer REST is a coordinated set of architectural constraints that attempts to minimize latency and network communication while at the same time maximizing the independence and scalability of component implementations. …. REST enables the caching and reuse of interactions, dynamic substitutability of components, and processing of actions by intermediaries, thereby meeting the needs of an Internet-scale distributed hypermedia system. Roy Fielding, Architectural Styles and the Design of Network-based Software Architectures • Roy is the principal author of the HTTP 1.0, HTTP 1.1, and URI specs • Co-founder of The Apache Software Foundation, co-author of the original Apache Web Server, author of libwww-perl

14. REST Easy • Agenda • Understanding REST • Using HTTP to build REST Applications • REST concepts and practices • Related technology • REST, WS-*, and SOA • Recommendations

15. Using HTTP to build REST Applications • What distributed systems protocol supports the implementation of REST applications? HTTP 1.1 aka RFC 2616 Media Types (multiple RFCs, IANA) URIs: RFC 3986 MIME: RFCs 2045/2046

16. Using HTTP to build REST Applications • What HTTP is not • Inevitable • Archaic • Poorly designed • Something to be suffered through • A transport protocol • The answer to everything • What HTTP is • Purposefully designed • REST conformant • Frequently abused • Wildly successful • An application protocol “Instead of embracing the people, principals, and technologies that gave rise to [the Web, software vendors] have chosen to subvert its history and to implant the ridiculous notion that it is ‘incapable of meeting the stringent demands of the business community.’” Ryan Tomayko, Motherhood and Apple Pie

17. Using HTTP to build REST Applications • HTTP: A brief introduction • A stateless, client/server, request/response application protocol • Clients make requests of servers using four main operations (uniform interface) • GET, POST, PUT, DELETE (there are others) • A request is sent to a specified URI (addressabilty) • An HTTP request/response is an envelope. • Inside the envelope are headers followed by a resource representation—if any (self-descriptive)

18. Using HTTP to build REST Applications • Anatomy of an HTTP GET GET / HTTP/1.1 Host: www.burtongroup.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3)... Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: [blah blah blah] Cache-Control: max-age=0 [ No entity body ] HTTP/1.x 200 OK Date: Thu, 17 May 2007 14:06:30 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 Cache-Control: private Content-Type: text/html; charset=utf-8 Content-Length: 16850 [ Response follows CRLF ] Headers Request Response General Entity Custom

19. Using HTTP to build REST Applications • Sample Application: A simple expense reporting system • Design a Web API that exposes all elements of an expense reporting systems • Users can create, edit, and delete entire expense reports and individual line items • Various views on the data will be allowed • Users must be authenticated • For example purposes • Users are not considered resources • No management hierarchy

20. Using HTTP to build REST Applications • The REST Recipe • Find all the nouns • Define the formats • Pick the operations • Highlight exceptional status codes

21. Using HTTP to build REST Applications • Find all the nouns • Everything in a RESTful system is a resource – a noun • If you find yourself creating verbs, noun-ify them • ApproveExpense  ExpenseApprovals • Every resource has a URI • Ideally just one (but don’t sweat it) • URIs should be descriptive • http://example.com/expenses;pending • Not a REST principle, but a good idea • Spend some time here, but don’t agonize over the perfect URI • URIs should be opaque • automated (non-human) clients should not infer meaning from a URI • URIs should be cool • Cool URIs don’t change

22. Using HTTP to build REST Applications • Find all the nouns • Use path variables to encode hierarchy • /expenses/123 • Use other punctuation to avoid implying hierarchy • /expenses/Q107;Q307 • /expenses/lacey,peter • Use query variables to imply inputs into an algorithm • /search?approved=false • You won’t need query variables as much as you think • Caches tend to (wrongly) ignore URIs with query variables • URI space is infinite (but URI length is not ~ 4K) • Don’t leak platform information • /expenses.php/123 /expenses?start=20070101&end=20071231 should be /expenses/20070101-20071231

23. Using HTTP to build REST Applications • Find all the nouns • Typically expressed as URI Templates • /{expense-id}/{line-item} • Starter List of Resources (not all inclusive) • Assumes base URI, e.g. http://expenses.example.com/v1

24. Using HTTP to build REST Applications • Define the formats • Neither HTTP nor REST mandate a single representation for data • A resource may have multiple representations • XML, JSON, binary (e.g., jpeg), name/value pairs • Schema languages are not required (if even possible) • Representations should be well-known media types (IANA registered MIME types) • Try and use “up-stack” media types • Makes your resources maximally accessible • XHTML or Atom instead of vanilla XML • You won’t always be able to do this, but you should try

25. Using HTTP to build REST Applications • Define the formats • Expenses (a collection of expense reports) • Will be presented as a list • Summary data only: report_id, submitter, total, status, link to detail • Therefore, need a format that presents a list of links • How 'bout XHTML! • <?xml version="1.0" encoding="UTF-8"?> • <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN“ "http://www.w3.org..."> • <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> • <head></head> • <body> • <ul class=“expense_reports"> • <li><a href="/users/bill/expenses/001">001</a>, Bill Smith, 1000.00, payed</li> • <li><a href="/users/pete/expenses/002">002</a>, Pete Lacey, 1500.00, submitted</li> • ... • </ul> • </body> • </html>

26. Using HTTP to build REST Applications • Define the formats: • Even better (tables are understood by spreadsheets) • <?xml version="1.0" encoding="UTF-8"?> • <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> • <head></head> • <body> • <table class="expenses"> • <tr class="expense_report"> • <td class=“report_id”> • <a rel=“expense_detail” href="/users/bill/expenses/001">001</a> • </td> • <td class=“submitter”> • <a rel=“employee_detail” href="http://employees.example.com/bsmith"> • Bill Smith</a> • </td> • <td class=“amount”>1000.00</td> • <td class=“status”>closed</td> • </tr> • ... • </table> • </body>

27. Using HTTP to build REST Applications • Define the formats • XHTML? Really? • It's XML, so it's parseable, XPathable, XQueryable, XCeterable • It's more accessible • Information available to anyone with a browser (IE issues aside) • Or an RSS reader or anything else that understands HTML • Has useful constructs for links, lists, name/value pairs, etc. • Graceful degradation from dedicated client to generic client • The class, rel, etc. attributes provide the necessary semantic value • No practical difference between: • <span class="amount">1000.00</span> • And • <amount>1000.00</amount> • Consider returning a stylesheet too; honored by browsers, ignored otherwise

28. Using HTTP to build REST Applications • A diversion on microformats • Extends XHTML to give semantic meaning to HTML tags • Extensions done with certain attributes: e.g. class, rel, rev • Formal microformats exist: http://microformats.org/ • geo: latitude and longitude • hCard: contact information • A dozen more • Often contrasted with the Semantic Web: RDF/OWL • Simpler • Leverages existing content • Strongly encouraged to use XHTML representations with official or custom microformats <div class="geo">GEO: <span class="latitude">37.386013</span>, <span class="longitude">-122.082932</span> </div>

29. Using HTTP to build REST Applications • Define the formats (continued) • Simple expense item: (straight XML this time) <ExpenseItem> <Date>2007-07-08</Date> <Description>Airfare</Description> <Amount>500.34</Amount> </ExpenseItem> • Simple expense report: (straight XML this time) <ExpenseReport xmlns:xlink="http://www.w3.org/1999/xlink"> <Submitter xlink:href=“http://employees.example.com/placey”> Peter Lacey </Submitter> <StartDate>2007-07-07</StartDate> <EndDate>2007-07-13</EndDate> <Status>SUBMITTED</Status> [expense items with links] <Total>9,999.99</Total> </ExpenseReport>

30. Using HTTP to build REST Applications • Pick the operations • HTTP has a constrained user interface (set of verbs/operations/methods) • GET • POST • PUT • DELETE • HEAD • OPTIONS (not widely supported) • TRACE (not significant) • CONNECT (not significant) • All of our resources will support GET

31. Using HTTP to build REST Applications • Pick the operations (just GET for now) “GET is one of the most optimized pieces of distributed systems plumbing in the world” Don Box (creator of SOAP) • GET returns a representation of the current state of a resource • GET is “safe” • Does not change resource state. • May trivially change server side state, e.g. log files • GET is “idempotent” • Multiple requests are the same as a single request • Might be impacted by concurrent operations • NEVER violate these rules

32. Using HTTP to build REST Applications • Highlight exceptional status codes • HTTP has more response codes than 200 and 404 – learn them • Information: 1xx, Success 2xx, Redirection 3xx, Client Error 4xx, Server Error 5xx • For GETs we’ll need: • 200 OK • 204 No Content • 404 Not Found • We’ll need more later

33. Using HTTP to build REST Applications • REST Frameworks • It is possible and legitimate to build REST systems with any HTTP-enabled application environment • Servlets, .NET, PHP, etc. • It’s just not fun • Few frameworks now, but more everyday • RESTlet (Java, open source) • Ruby on Rails (Ruby, open source) • Django (Python, open source) • CherryPy (Python, open source) • JSR 311/JAX-RS (Java) • Astoria (.NET, Microsoft labs project) • Project Zero (Groovy/PHP, IBM incubator project) • .NET 3.5 (aka Orcas, unreleased)

34. Using HTTP to build REST Applications • RESTlet • An open source Java framework for building RESTful applications • Akin to the servlet API, but maps directly to REST concepts • Classes for resources, representations, routers • Can run standalone or in a servlet engine • Provides a client API too

35. Using HTTP to build REST Applications • RESTlet Architecture • Components manage virtual hosts and applications • Applications manage a set of related RESTlets • RESTlets are classes that respond to a uniform interface • RESTlet Framework is composed largely of RESTlets

36. Using HTTP to build REST Applications RESTlet Architecture Component Application 1 Business Resource Router Domain Object Guard Business Resource Domain Object Virtual Host 1 Finder Root Business Resource Application N Virtual Host N Domain Object Business Resource Router Finder Domain Object Business Resource Root

37. Using HTTP to build REST Applications • RESTlet Architecture • Representation variants

38. Using HTTP to build REST Applications • Content Negotiation • A single resource can have multiple representations • XML, PNG, JSON, etc. • English, Japanese, etc. • Client can ask for a particular representations via request headers • Accept: text/xml,application/xml,application/xhtml+xml, • text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 • Accept-Language: en-us,en;q=0.5 • Accept-Encoding: gzip,deflate • Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 • “q” = relative preference (0-1)

39. Using HTTP to build REST Applications • Content Negotiation • Doesn’t work in practice • Most clients don’t expose a means to set Accept headers • Some clients don’t support Accept headers at all • Best Practice: Use Accept headers and expose the content type in the URI • http://expenses.example.com/123.xml • http://expenses.example.com/123.xhtml • http://expenses.example.com/123.json • Dots preferred over slashes: http://expenses.example.com/123/xml

40. Using HTTP to build REST Applications • Conditional GET • “Give me a resource representation, but not if it hasn’t changed since last time I asked.” • Saves bandwidth, client processing, and (possibly) server processing • Two mechanisms: Last-Modified and ETag Client Server GET /expenses;pending 200 OK Last-Modified: Sun, 21 Jan 2007 09:35:19 GMT [response] GET /expenses;pending If-Modified-Since: Sun, 21 Jan 2007 09:35:19 GMT 304 Not Modified

41. Using HTTP to build REST Applications Conditional GET: ETag Mechanism Client Server GET /expenses;pending 200 OK ETag:”some UID” [response] GET /expenses;pending If-None-Match:”some UID” 304 Not Modified

42. Using HTTP to build REST Applications • Conditional GET • Always support conditional GETs • Servers should use both Last-Modified and ETag • Common to use hashes for Etags • When creating ETags consider the expense of generating the representation and the ETag • Use “deep” ETags • Build the notion of “changed” deep into your app • E.g., a timestamp column in the database “This stuff matters” Joe Gregorio (co-author of Atom and APP)

43. Using HTTP to build REST Applications • Caching • Caching reduces latency and network traffic • Only (non-SSL) GETs are cached – not POSTs etc. • Two kinds of caches • Browser (user agent) • Proxy • Largish sites really should use a caching proxy server • You may be and not even know it • Remember, there can be caches all along the way. • Your application must know how to control how its content is cached • Otherwise, defaults may be used • Don’t cache sensitive info

44. Using HTTP to build REST Applications • Caching • Important cache-control headers • Validate = Conditional GET • Example: • Cache-Control: max-age=3600, must-revalidate

45. Using HTTP to build REST Applications • The Writable Web • Three principal state changing verbs • PUT: Not safe, idempotent • Create a new resource at the specified URI • Update existing resource at the specified URI • Typical response: 201 Created or 400 Bad Request • DELETE: Not safe, idempotent • Remove the resource at the specified URI • Typical response: 200 OK • POST: Not safe, not idempotent • Create a subordinate resource • Create child of resource at specified URI • Create new resource from factory resource at specified URI • Typical response: 201 Created + Location or 400 Bad Request • Append representation to existing resource • “Process this”

46. Using HTTP to build REST Applications • The Writable Web: • When creating new resources • Use POST if the server chooses the URI • Use PUT if the client chooses the URI • "Process this" • POST does “something” and returns “something” • Usually masking an RPC call ("overloaded POST") • Don't use casually • "Process this" POST sometimes valuable: • Resource factories • Sometimes you just want a verb • If you find yourself creating resources that can’t be retrieved with GET, reexamine your design http://expense.example.com/pay_expense • But don’t sweat it, if it gets you out of a design jam

47. Using HTTP to build REST Applications • The Writable Web • Some HTTP clients, servers, or middleware don’t understand PUT & DELETE • Including HTML forms and some firewalls/proxy servers • Valid to overload POST to mimic PUT and DELETE in this case • Example: POST /123/line+items/2?_method=delete

48. Using HTTP to build REST Applications The Writable Web

49. Using HTTP to build REST Applications • Asynchronous Operations • E.g.: Cut a check for an expense. Check processing system is remote. Accepts requests on a queue. • Expose the queue as a resource • Use the “202 Accepted” response and create a new “job” resource for later querying POST /payment+queue HTTP 1.1 Host: expenses.example.com <representation of a payment> ► 202 Accepted ► Location: /payment+queue/4567

50. Using HTTP to build REST Applications • Asynchronous Operations • Poll job resource for status GET /payment+queue/4567 HTTP 1.1 Host: expenses.example.com ► 200 OK ► Content-Type: text/plain ► queued payment of $100 to Pete Lacey (later) ► dequeued payment of $100 to Pete Lacey (later) ► Check for $100 paid to Pete Lacey