Applied Web Services: EBay, Google, and Amazon

1. Real World Web Services(was Applied Web Services) Christian Gross Mailing@devspace.com

2. Introduction: Christian Gross • Trainer, Consultant, Mentor • Open / Shared Source / Binary technologies • Technologies (Web Services, Apache / XML, Java, C++, .NET) • Website • http://www.devspace.com • Author: • Applied Software Engineering Using Apache Jakarta Commons • “Open Source for the Windows Administrator” • Upcoming “(Re)Learning OOP using .NET Patterns” • Upcoming “AJAX/REST Patterns and Best Practices” • Contact: mailing@devspace.com

3. Why Rename Of Session? • A funny thing happened… • I was late with this tutorial, and I thought I had everything ironed out • HA! • Web Services has changed in its implementation • Originally this tutorial was about using SOAP, and WSDL to interact with EBay, Google, and Amazon • But the big players EBay, Google, and Amazon have changed tack • Some of it was already happening, but now the ball is rolling!

4. Agenda • Outline of what Applied Web Services means • Defining and illustrating what EBay, Google, and Amazon are doing • Code walk through during entire tutorial • mailing@devspace.com

5. What Is Applied Web Services? • Applied Web Services is when two computers communicate to exchange data that only the computers are interested in • Sounds like Theory of Web Services? • Difference is that the computers are made to work together in a heterogeneous environment • Many pieces of Web Services is based on unknown theory that may or may not work, as people are churning out the specifications • Many internal web service applications are based on the same platform • Kind of ironic, no? Sort of like buying a fleet of Mercedes and getting them repaired at a Ford dealership. Misguided “just in case” thinking • Even though interoperability is an issue, I wonder if it is not a red herring argument

6. Applied Web Services • The big dream of web services is to have a meta-data with appropriate specifications results in two computers being able to transfer data • Wrong idea, and will never happen! • Better idea is to apply web services with specific domain knowledge, with specific protocols • Challenge (you will get one of my books your choice): • Easy: What is a six-pack • Medium: What is a two by four? • Hard: What is a two-four? • The problem is that people like to optimize and use path of least resistance!

7. How Does It Affect You? • Most of the time your web services will be fully under your control • Internal, sub-contractor, etc • But for EBay, Google, and Amazon and other corporations you are not in control • Think of it as computing across the Internet • YOU MUST BE CONNECTED ALWAYS! Not optional and introduces a new paradigm • Involves using Web Services as a consumer and not service provider • Of course you can offer your own Web Service (eg create a service that combines the best Entertainment System)

8. Different Protocols ATOM Consumer Provider RSS REST SOAP

9. RSS Rich Site Summary (RSS) is a XML based web syndication mechanism used by newsites, and weblogs There are three versions 0.91 (Rich Site Summary), (RDF Site Summary) 0.9/1.0, and 2.0 (Really Simple Syndication) Allows a website to syndicate information Can be used for personal diary, version updates, news items Used to inform clients of things that are happening Common software utility is called an aggregator that combines the streams and presents a new stream RSS 2.0 allows extensions for additional functionality Microsoft using it for ordered updates

10. RSS Format <rss version="2.0" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:admin="http://webns.net/mvcb/" > <channel> <title>Blogger Jacks</title> <link>http://www.devspace.com</link> <description>News and Politics for the Software Industry</description> <dc:language>en-us</dc:language> <dc:creator>Christian Gross</dc:creator> <dc:date>2005-06-21T11:24:09+02:00</dc:date> <admin:generatorAgent rdf:resource="http://nanoblogger.sourceforge.net" /> <lastBuildDate>Tue, 10 Jun 2003 09:41:01 GMT</lastBuildDate> <docs>http://www.devspace.com/tech/rss</docs> <generator>Weblog Editor 2.0</generator> <managingEditor>editor@example.com</managingEditor> <webMaster>webmaster@example.com</webMaster> <item> <link>http://www.devspace.com/archives/2005/06/22T09_30_00.html</link> <title>Backward Thinking and Forward Thinking</title> <dc:date>2005-06-22T09:30:00+02:00</dc:date> <dc:creator>Christian Gross</dc:creator> <description><![CDATA[Some text]]></description> <pubDate>Tue, 03 Jun 2003 09:39:21 GMT</pubDate> <guid>http://www.devspace.com/ archives/2005/06/22T09_30_00.html</guid> </item> </channel> </rss> Channel Description Item Description

11. ATOM Controversial because it is meant to replace RSS as it does the same thing as RSS Meant to keep things simple when you need to retrieve, update, and modify streams RSS is only the viewing, not the updating ATOM is based on XML, and only one protocol HTTP Includes Syndication, Publishing, and Auto-Discovery Can be used to create “resources” and reference information

12. ATOM Feed (http://en.wikipedia.org/wiki/Atom_%28standard%29)

13. REST == Representational State Transfer • REST is an architectural style, not a standard • "Representational State Transfer is intended to evoke an image of how a well-designed Web application behaves: a network of web pages (a virtual state-machine), where the user progresses through an application by selecting links (state transitions), resulting in the next page (representing the next state of the application) being transferred to the user and rendered for their use." - Dr. Roy T. Fielding • REST emphasizes • Scalability of component interactions • Generality of interfaces • Independent deployment of components • Support for intermediaries

14. Key Principles of REST - Peter Drayton • "Identification of resources" • Resources are anything that can be named • Naming authority assigns an identifier to a resource • "Manipulation of resources through representations" • Representations capture current/intended state of resources • Representations are transferred between components • Representations often contain links to related resources • "Self-descriptive messages" • Resource identifiers, representation data formats, control data • "Hypermedia as the engine of application state" • Servers are stateless, messages are independent • Clients maintain state (representations) & traverse links • Induces scalability, generality, evolvability, extensiblity

15. REST Process View - Roy T. Fielding $ $ $ $ $ $ $ $ • Layered Client-Server • Uniform Interface (like Pipe and Filter) • Stateless, Cacheable Communication • Optional Code-on-Demand

16. SOAP • Initially used for RPC (Remote Procedure Call) • Has undergone quite a few revisions in a very short period of time (0.9, 1.0, 1.1, 1.2) • Part of the XML Protocol working group • Based on the concept of an envelope encapsulating a document • Like snail mail, envelope and letter within

17. SOAP Theory • Specification is 1.2 that is split into four documents • Part 0: Primer, which is easy to read and understand • Part 1: Messaging Framework, which provides the basics of the SOAP message structure. Not so easy to read. • Part 2: Adjuncts, defines a number of “extensions” to the SOAP protocol. When read must understand Part 1. • Specification Assertions and Test Collection, used to define the test conditions to validate a client or server implementation. Generally only useful for authors of SOAP implementations • For end user programming purposes Part 0 is the only specification that interesting enough to be read

18. SOAP Theory 1.2: Message • Make an HTTP Request (SOAP in payload) • HTTP is an example of a SOAP Binding • Binding and Message are now two separate aspects of the SOAP specification POST /StockQuote HTTP/1.1 Host: www.stockquoteserver.com Content-Type: text/xml Content-Length: nnnn SOAPMethodName: Some-Namespace-URI#GetLastTradePrice <SOAP:Envelope xmlns:SOAP="urn:schemas-xmlsoap-org:soap.v1"> <SOAP:Body> <m:GetLastTradePrice xmlns:m="Some-Namespace"> <symbol>DIS</symbol> </m:GetLastTradePrice> </SOAP:Body> </SOAP:Envelope>

19. WSDL Theory: Introduction • Web Services Description Language (WSDL) • A way of describing your web service • May or MAY NOT be SOAP based • Extensions include SOAP or HTTP • Can be anything including a proprietary protocol Service interface (Abstract definition) Messages Operation Interface Service implementation (Concrete definition) Binding Service Endpoint

20. WSDL The Good and The Bad • WSDL 1.1 != WSDL 2.0 • Major changes and there is no compatibility • Rich Salz has a major beef (http://webservices.xml.com/pub/a/ws/2004/11/17/salz.html) • His point is both valid and overblown • Moves back into the COM, IDL, CORBA way of specifying an interface • Difference is that everything is expressed using XML • The problem that WSDL has its requirement to “describe the world” • Requires to be very loose and open ended thus making it hard to be 100% specific. • What about the future? WSDL needs to address the future and whatever it encompasses

21. What Do You Need To Know? • Good question, since there are too many choices • You need to know that you should not lock yourself into a single protocol • Understand your Web Services platform • Amazon, Google, and EBay have extensive (mostly) help on how to get going with the API’s. • Know that the Web Service is not yours! • This means you cannot simply resell, call, manipulate or do what you want • There are rules of conduct defined by the Web Service provider • If you don’t agree with them TOUGH BANANAS! • The data is not yours and you may not use it as yours • In other words no permanent caches or caches in general

22. Three Major Web Service Platforms • Amazon: Buying of “things” • New Web Services to send messages and search websites • EBay: Buying and selling of “things” in an auction scenario • Comprehensive platform to do many things • Google: Doing many “things” • There are others, but they are beyond the scope of this talk

23. What Can You Do? • Buying, selling, and searching of products using EBay SDK • Buying of products using Amazon SDK • Searching of the Internet using Google and Amazon SDK • Simple Queuing Services using Amazon SDK • Atom / Blogger API using Blogger SDK (Google company) • MAP API for downloading and searching

24. EBay SDK • Developer information is available at http://developer.ebay.com • Two main developer issues • Application creation • Certification • You can easily develop an application in different languages and environments • Supported SOAP, Raw XML, and moving towards REST (60 API’s approx) • Multiple languages, Java, .NET, PHP, Perl, Python, etc • Data is being moved to UTF-8 for simpler international consumption • Need a web service toolkit that supports https (SSL) • If keys are lost there are issues with respect to hackers doing things on you behalf

25. EBay SDK (cont.) • All initial development is carried out in a Sandbox • Is a mirror of the EBay America website • The is American only meaning anybody can use it, but the data is presented using American notations • Can use fake data, and processing will be real, but is not actually carried out • There are two sets of keys sandbox and production keys • Production keys are only given out when your application is certified • Are constantly monitored for which API’s are used and when

26. EBay Certification • All applications must be certified before they can be used in production • Production keys are given when application is certified • Certification requires filling out a formula and answering some questions • Generally a 5-7 business day process • Certification means: • Having used EBay in the sandbox (active monitoring) • Specifying and having to update which API’s can be used • Used by EBay to assess future hardware requirements • Implementing the EBay guidelines, eg not storing data locally or indicating where the data came from • Not yet implemented “charge on behalf”

27. Amazon SDK • Developer reference is found at: • Surf to http://www.amazon.com • Click on “See More Stores” link • Click on “Web Services” link • A developer key is required • Supported is both REST and SOAP • Language and platform support is not an issue because all API’s are supported regardless of technology • Platform or language must be able to understand HTTP, and XML • Free platform, no certification, but limits on usage • Can get paid using affiliate service

28. Amazon SDK (cont.) • Amazon has three main services • Amazon store: Used to browse and search for products • Amazon Alexa Service: A search engine used to search for information • Amazon Simple Queuing Services: Used to send queued packets, essentially an “email” service without the email • Amazon was one of the first to implement REST, and offers both (SOAP, and REST) • Geared towards using XSLT • Amazon provides additional tools such as: • XML Scratch Pad: A tool used to create Amazon requests in XML without having to code the XML yourself • XPath XSLT Tool: A tool based on the Amazon Web Services used to generate XPath and XSLT documents • XSLT is often used in conjunction with REST and is very popular

29. Amazon Store • Searches the Amazon store • Can specify a local store (US, DE, FR, CA, JP, UK) • Can search Amazon zShops • Can perform minimal, or extensive searches • Minimal as an example retrieves product information and cost • Extensive searches include comments, image references, etc • Can implement a remote shopping cart • Check out must be performed by Amazon.com • Can search wishlists of other people • Ability to perform multiple actions in one request

30. Amazon Alexa Search • Can execute a search using the http://www.alexa.com search engine • There are multiple ways to search: • Regular search like a Google search • Category search: Alexa has a set of categories that can be used to narrow down the search • Web map, and URL info makes it possible to retrieve the information regarding a specific URL, such as reference count, and popularity

31. Amazon Simple Queuing • An odd service, but makes it possible to queue information in a data store on the Internet • No idea what the intent is • Used for distributed applications that would like to share information • Has a limitation in that queue elements can only be 4KB

32. Google API’s • Google has gone from a single API for its search engine to a repository or things that you can do • Developer information found at the URLhttp://code.google.com • All of this is still in beta form and requires registration • You cannot make commercial usage of any API’s unless you get written permission from Google

33. Google Search • A developer token is required • Can only be used for personal non-commercial purposes • For anything else you must write to Google and ask • Only a SOAP interface and can be used by anything that can process a WSDL file • Is considered a beta service at the time for trial purposes • Is used to search the Internet, perform spelling, etc

34. Google AdWords • Used to manage AdWord accounts • Can be used to manage keywords dependent on your applications, or databases • Only a SOAP interface, but a WSDL file is provided

35. ATOM Feeds • Blogger is a company purchased by Google and used to manage blogs • Need to use the ATOM API to add, delete and manipulate entries in a blog • GMail is a new web based mailing system like Yahoo mail • Is used to read new emails only, and not manipulate them • Google Groups does not have an ATOM feed, but you can use RSS • All of these information feeds can be used by you for aggregation purposes

36. Maps and Google Earth • Maps, and Google Earth are two new products that make it possible to research our earth • Google Earth is a locally installed application, but it can be enhanced by using KML (Keyhole Markup Language) • KML is used to define points, lines, and images in the Google Earth client • Maps is a web based application that can be enhanced by interleaving user information with maps information • Is intended to be used client side in a JavaScript, but is not necessary

37. In Conclusion • The future is the NET and major companies are making major investments • Instead of a single protocol there are multiple protocols like ATOM, RSS, SOAP, WSDL, and REST • The data and service is not yours, you are a client and subject to the terms of the web service provider • It is a good way to provide pre-packaged functionality • There are other publicly available web services at the URL http://www.xmethods.net

38. Thanks Questions?