Chapter 5 Web Remoting Techniques – the A in Ajax

Chapter 5Web Remoting Techniques – the A in Ajax

Chapter Objectives • Discuss Web Remoting • Introduce the XMLHttpRequest object and how to make asynchronous requests. • Discuss the hidden IFrame technique and give an example. • Describe the HTTP Streaming technique.

Web Remoting • Web Remoting is a term used to categorize the technique of using JavaScript to directly make an HTTP request to the server and process the response. • Traditionally, Web browsers initiate HTTP requests, not scripts. The classical function of a Web browser is based on a synchronous request/response model where HTTP requests are made in response to the user clicking links, submitting a form, or typing in a URL. • The browser processes the request by discarding the current page (including any running scripts), downloading a new page from the server, and then loading the new page in the browser for the user to view.

Web Browser User submits form user waits… user waits… <html> <html> Web Server Process request User clicks link User enters URL Process request • This is a time-consuming process, especially when you don’t need to display a whole new page. • Often you only need to send a small amount of information to the server, or update a portion of the current page with a little data from the server.

To work around the limitations of this page-driven paradigm Web developers have constructed techniques to exploit URL-supporting HTML tags and CSS properties in non-traditional ways. • Every tag with a src or href attribute that can be used without reloading the entire page is a candidate, including <img>, <script>, <link>, <frame>, and <iframe>. When a script sets the src or href property to a URL the browser performs an HTTP GET request to download the content of the URL, without reloading the page. • You can exploit this fact to send information to the server by encoding that information in the URL query string. • You can additionally have the server set a cookie in the response with information you need.

XMLHttpRequest • While the techniques that use HTML tags (also called remote scripting) are often useful, they are, strictly speaking, a misuse of their intended design, and can sometimes be complicated to get working properly and uniformly across browsers. • Alternatively, the XMLHttpRequest object provides full support for making asynchronous HTTP requests including HEAD, POST, and GET, and can, despite its name, handle responses in plain text, XML, or HTML. • This allows the Web application to process user’s input and respond with data from the server without having to make the user wait for an entire page to load.

<html xmlns="http://www.w3.org/1999/xhtml"> <head>…</head> <body> <form id="carSearchForm"> <label for="year">Year:</label> <input id="year" name="year" type="text" size="5" /><br /> <label for="make">Make: </label> <select id="make" name="make"> <option value="">Select Make...</option> … </select><br /> <label for="model">Model: </label> <select id="model" name="model" disabled="disabled"> <option value="">Select Model...</option> </select><br /> <label for="zip">Zip: </label> <input id="zip" name="zip" type="text" size="11" /><br /> <div id="buttonRow"> <button type="submit" disabled="disabled">Search for my next car!</button> </div> </form> </body> </html> • As a simple example, consider a Web application that allows users to search for a specific model of car being sold in their area. It provides a search form and validates the input.

Form Processing without XMLHttpRequest

Step 1: The user enters the URL of your Web site into their browser and the browser loads the page with the form for the first time. • Step 2: The user selects a make from the list and the event handler is triggered to submit the form to the server to look up associated models. The server responds with a whole new page including the form, the selected make, and the associated models. • Step 3: The user has populated the zip and left that field, which triggered an event handler that submitted the form for validation. The server determines the zip is invalid and responds with an entire new page including the form, the selected and populated fields, and a validation error message.

Form Processing with XMLHttpRequest

Step 1: The same as before – the user navigates to your Web site. • Step 2: After the user selects a make, the event handler uses an XMLHttpRequest to send only the selected make to the server, which responds with only the list of associated models, which are used to update the display. • Step 3: After the user populates the zip field, the event handler uses an XMLHttpRequest to send only the entered zip to the server, which responds with only a validation error message that is, in turn, displayed to the user. • The response is much faster because the client and server are sending very little information back and forth, and the browser doesn’t have to re-render the entire page just to show small changes in information.

Microsoft was the first to implement asynchronous request functionality in Internet Explorer version 5.0, released in March 1999. Microsoft first implemented the object as XMLHTTP, an ActiveX component in an XML library called MSXML. • Mozilla ported the idea to their Mozilla 1.0 browser, released in May 2002. Mozilla decided to call their object XMLHttpRequest and make it native to the browser’s JavaScript interpreter. • Apple followed later with a native implementation in Safari 1.2 (February 2004) and Opera added a native implementation in version 8.0 (April 2005) • Internet Explorer 7 (October 2006) implemented a native replacement to XMLHTTP called XMLHttpRequest.

Creating an XMLHttpRequest Object Using XMLHttpRequest is essentially a three-step process. • Create an XMLHttpRequest object. • Configure and submit your HTTP request to the server. • Process the server’s response. • Because Microsoft’s implementation prior to IE 7 was an ActiveX object, you need to use some conditional logic to create an XMLHttpRequest instance.varxhr;if (window.XMLHttpRequest) {xhr = new XMLHttpRequest();} else if (window.ActiveXObject) {xhr = new ActiveXObject("Msxml2.XMLHTTP.6.0");} else { throw new Error("XMLHttpRequest not supported."); }

Several versions of Microsoft’s MSXML library are in existence and multiple versions can be installed on the same computer. • The different versions of MSXML libraries are identified by a ProgID (ie, “Msxml2.XMLHTTP.6.0”, “Msxml2.XMLHTTP.3.0”). You pass one of these ProgIDs to the ActiveXObject() constructor to create an instance. • The only versions of MSXML that Microsoft suggests using are 3.0 and 6.0. Version 6.0 is preferred and 3.0 is a fallback for older systems. All other versions have problems with security, stability, or availability. Windows Vista ships with MSXML 6.0, which can also be installed on Windows 2000, Windows Server 2003, and Windows XP.

Sending an HTTP Request • Once you have created an instance of XMLHttpRequest, the API for using it is the same in all browsers. • Using an XMLHttpRequest object is a multistep process. First initialize the object to specify what URL to request and how to use the object, then call the send() method to actually make the request, and finally, process the response. • Call the open() method to specify the URL and whether you want to make a synchronous or asynchronous request. • Call the setRequestHeader() method to specify any HTTP headers to be sent with the request. • Set the onreadystatechange property to a function that will be called as the state of the request changes (and process the response when the readyState is 4.

var xhr = …Create an XMLHttpRequest object // Define the readyState event handler to process the response. xhr.onreadystatechange = function() { if (xhr.readyState == 4) { if (xhr.status == 200 || xhr.status == 304) { var models = xhr.responseText; …Do something with the response data } else { // Some error occurred, so notify the user. alert("Failed to get car models. Server response: " + xhr.status + " " + xhr.statusText); } } } // Tell XMLHttpRequest what URL we want to GET. xhr.open("GET", "http://blah/getCarModels.php?make=" + encodeURIComponent(selectedMake), true); // Send the request. Pass null if not sending any data. xhr.send(null);

Web Server Web page using Ajax 3 XMLHttpRequest Server Resource 5 6 2 onreadystatechange = function() { // Update model <select> } 4 1 7 Database Web Browser Server • Interaction when user selects car make and Ajax is used to retrieve car model and populate drop down list:

XMLHttpRequest void open(DOMString method, DOMString url, boolean async, DOMString user, DOMString password)Initializes the request.method (required): A string that defines the HTTP method of the request, which essentially describes the operation to be performed on the resource identified by the URL of the request. Typical HTTP methods are HEAD, GET, and POST.url (required): A string defining the target URL of the request.async (optional): A Boolean value indicating if the request should be made asynchronously. By default the value is true, which processes the request asynchronously.user (optional): A string identifying a user for a Web server that requires authentication.password (optional): A string identifying the user’s password.

void send(Object data)Sends the request to the server. If the request was configured to be asynchronous then this method returns immediately; otherwise it blocks until the response is received. The optional parameter is sent as part of the request body. The parameter can be a DOM Document object, an input stream, or a string. The parameter is usually only used when sending a POST request with form data. onreadystatechangeAn event handler that is invoked as the request changes state. An XMLHttpRequest object progresses through five states as the request is being processed.

readyStateAn integer value that is set to the current state of the request. The browser changes the readyState of an XMLHttpRequest object as the request is being processed, which causes the onreadystatechange event handler to be called allowing you to react to the change in state. Usually, the only state you care about is 4 Completed.0 Uninitialized: The open() method hasn’t been called yet.1 Loading: The open() method has been called, but the send() method has not.2 Loaded: The send() method has been called, but the server has not responded yet.3 Interactive: A partial response has been received.4 Completed: The complete response has been received and the connection is closed.

responseText: The body of the server’s response as a string of text. The server can respond to the request with plain text, HTML, XML, whatever. As long as the request was successful, this value will get set to the response data in string form no matter what format came back. responseXML: If the response is an XML document (and the response Content-Type header is set to “text/xml”), then the XML will be parsed into a DOM Document object and assigned to this property. status: An integer value containing the HTTP status code of the response. statusText: A string value containing the text description of the status code (“OK” for 200, “Found” for 302, etc.).

Sending a POST Request • If you want to send some data to the server in the body of the request, you can pass it to the send() method. You simply have to concatenate the form field name/value pairs into a URL encoded string that you pass to the send() method.// Set up a POST request.request.open("POST", "http://localhost/ajax-ch5/search.php", true); // Set the proper Content-Type.request.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded'); // Encode the form.var searchForm = document.getElementById("carSearchForm");var encodedForm = RequestHelper.encodeForm(searchForm); // Send the form to the server.request.send(encodedForm);

Processing the Response • The HTTP response data is identified by the Content-Type header, which specifies a MIME (Multipurpose Internet Mail Extensions) type. • Theoretically, you can respond to an XMLHttpRequest in any format that the browser supports, but the XMLHttpRequest object only has special handling for XML data. • The browser will automatically parse a response that is well-formed XML with the Content-Type header set to "text/xml", "application/xml", or any other MIME type that ends with "+xml" (like "application/xhtml+xml"). If the XML data is not well-formed or the Content-Type header is not set properly, then the XMLHttpRequest responseXML property will not contain a DOM Document object.

If the response does not contain XML, then the browser assigns the HTTP response body to the XMLHttpRequest responseText property and it’s up to you to parse the data, if needed. • The most common response formats used in Ajax development are XML ("text/xml"), HTML ("text/html"), plain text ("text/plain"), JavaScript ("text/javascript"), and JavaScript Object Notation ("application/json"). • Plain text data is good for small pieces of data like a validation message. HTML data is useful if you need only need to update content within a single DOM node – meaning you can use innerHTML and not have to parse the HTML manually. XML and JSON are good for transferring several pieces of data that are to be used in various parts of the page or for calculations.

HTML Response • An HTML response is great when you can use the innerHTML property. Otherwise you have to parse the HTML manually, which is painstaking. • The innerHTML property is easy to use and the browser processes the change fast. Just assign a string of HTML code to the innerHTML property of a Node and the browser replaces the Node’s content with the given HTML – no need to use the DOM API.var models = request.responseText;var modelSelect = document.getElementById("model");modelSelect.innerHTML = models;

XML Response • XML is a versatile and well-supported choice for your response format. Practically all server-side languages have support for processing XML and the browser will automatically parse an XML response into DOM objects and make the data available via the XMLHttpRequest’s responseXML property. • Using XML, you have precise control over how your response data looks. On the client, you can use various parts of your XML response to update various parts of the HTML. • The main drawback to using XML is the client-side code that uses the DOM API to update parts of the loaded document can become quite verbose.

The browser will only parse an XML response and assign it to the XMLHttpRequest responseXML property if the Content-Type HTTP header is set to either "text/xml", "application/xml", or a type suffixed with "+xml". If you forget to set the correct Content-Type header on the server, then responseXML will be empty.// Create an instance of XMLHttpRequestvar request = RequestHelper.createXHR(); request.onreadystatechange = function() { if (request.readyState == 4) { if (request.status == 200 || request.status == 304) { // Response is XML parsed into a Document object.var response = request.responseXML; …Do something with XML } } }

Plain Text Response • When you want to do something as simple as have the server validate a field and respond with an error message (i.e., "*Must be a 4-digit year"), then the easiest response format is just plain text. • As an example, assume we have the following HTML<input id="year" name="year" type="text" size="5" /><span id="yearMsg" class="error"></span><br />Then the response processing could look as simple as thisvar request = RequestHelper.createXHR(); request.onreadystatechange = function() { if (request.readyState == 4) { if (request.status == 200 || request.status == 304) {var response = request.responseText;document.getElementById("yearMsg").innerHTML = message; } } }

JSON Response • The text returned from the server and made available via the responseText property of XMLHttpRequest can be JavaScript code. • JavaScript provides a built in eval() function that will take a string, interpret it as JavaScript, and return the result. • JSON is defined as a lightweight data-interchange format that is based on a subset of the JavaScript language, namely JavaScript object and array literals. • JSON is much less verbose than XML, which explains why it is touted as the “fat-free alternative to XML”.

An object in JSON is declared as a set of name/value pairs enclosed in braces, {}. Each name in the object is a string followed by a colon (:) and the name/value pairs are separated by a comma (,). The value can be a string, number, object, array, true, false, or null. • An array in JSON is an ordered collection of values enclosed in brackets, []. Each value in the array is separated by a comma (,).{ "error": { "field": "year", "message": "*Must be a 4-digit year“ }, "make": { "name": "Jeep", "models": ["Commander", "Compass", "Grand Cherokee", "Liberty", "Wrangler", "Wrangler Unlimited"] }}

Example:// Create an instance of XMLHttpRequestvar request = RequestHelper.createXHR(); request.onreadystatechange = function() { if (request.readyState == 4) { if (request.status == 200 || request.status == 304) { // Response is a JSON string.var jsonResponseString = request.responseText;// Convert the JSON string to an object. var response = eval("(" + jsonResponseString + ")"); …Do something with the data in the JavaScript object } } }

Passing a string to eval() has some security implications. The JSON specification just deals with data, but there is nothing to stop you from defining a function in the JSON string – it will work even though it’s not part of the JSON spec. To close this security hole, Douglas Crockford (and others) have written JSON parsers that only recognize JSON syntax and reject text that contains anything else, like functions and assignments. Douglas Crockford’s parser is simply called JSON. Using the JSON parser we could replace the line using eval() with the following. If the JSON string contains something like a function, the parser throws an error.// Throws a SyntaxError if the JSON string is invalid.var response = JSON.parse(jsonResponseString);

JSON can be used to not only send data from the server to the client, but also from the client to the server. All you need is a processor on the server to convert your JSON data into the programming language of your choice. Fortunately processors for many different languages already exist to convert your data both to and from JSON. For example, Douglas Crockford’s JSON parser also has a method to convert a JavaScript object to a JSON string, which you can then send to the server. For example, we could create a JavaScript object containing the selected car make and the entered year, then use JSON to convert it to a JSON string like this.var makeSelect = document.getElementById("make");var data = {year: document.getElementById("year").value, make: makeSelect.options[makeSelect.selectedIndex].value};var jsonData = JSON.stringify(data);

Timing out a Request • A deficiency of XMLHttpRequest is the lack of a timeout mechanism. However, the XMLHttpRequest object does provide the abort() method to cancel a request.var request = RequestHelper.createXHR(); var abortTimer = setTimeout(function() {request.abort();alert("Failed to get car models. Please try again.");}, 15000);request.onreadystatechange = function() { if (request.readyState == 4) {clearTimeout(abortTimer); if (request.status == 200 || request.status == 304) { … } } }

Hidden IFrame • Long before there was wide-spread browser support for, or use of, an XMLHttpRequest object, developers were using hidden frames to make requests to the server without reloading the page the user is viewing. • You can hide a frame using CSS and use JavaScript to dynamically load content into the hidden frame without the user knowing. • An <iframe> can be placed anywhere between the <body> tags, and so it becomes embedded in the content of the HTML page. • A limitation of the XMLHttpRequest object is that you cannot use it to upload a file, so, the only alternative is to use a hidden frame.

Typical Data Flow using a Hidden IFrame

As an example, suppose we want to create another form for the car sales Web application that allows the user to register a car they want to sell. This registration form will allow the user to enter information about the car and upload a picture of the car, which the application will use to make a listing for the car that other users can browse. The figure in the previous slide shows how the new registration form may look and illustrates the typical data flow when using a hidden <iframe>. • In step 1 the main page uses JavaScript to pass data to the page loaded in the IFrame and sets the source URL of the IFrame. When the IFrame’s source URL is changed, it automatically sends a request to the Web server (step 2). • In step 3 the Web server responds with an HTML page containing any data the client needs. • In step 4, JavaScript loaded in the IFrame page updates the main page.

<html xmlns="http://www.w3.org/1999/xhtml"> <head> … <script type="text/javascript" src="upload.js"></script> </head> <body> <form id="carEntryForm" name="carEntryForm" method="POST" enctype="multipart/form-data" action="registerCar.php" target="uploadFrame"> … <label for="picture">Picture:</label> <input id="picture" name="picture" type="file" /><br /> <div id="buttonRow"> <button id="submitBtn" type="submit"> Sell my car! </button> </div> </form> <iframe src="about:blank" id="uploadFrame" name="uploadFrame"></iframe> </body> </html>

function init() { var submitBtn = document.getElementById("submitBtn"); var origBtnText = submitBtn.firstChild.nodeValue; var form = document.getElementById("carEntryForm"); form.onsubmit = function() { submitBtn.disabled = true; submitBtn.firstChild.nodeValue = "Uploading Information..."; return true; }; var iframe = document.getElementById("uploadFrame"); iframe.onload = function() { if (iframe.href != "about:blank") { submitBtn.firstChild.nodeValue = origBtnText; submitBtn.disabled = false; } return true; }; } // Execute init() after the page loads. window.onload = init;

HTTP Streaming • The standard HTTP model is based on the Web browser pulling data from the Web server. • To achieve a push model where the server pushes data to the browser, one of two techniques is typically used, although a combination can also be done. The first technique involves the normal request from the browser, followed by multiple responses from the server using the same long-lived HTTP connection. This technique is sometimes called page streaming.

Page Streaming Communication

Some drawbacks to page streaming are: (1) The browser accumulates objects, which uses up memory and could eventually cause the interface to bog down; (2) HTTP connections will inevitably fail, so you must have some plan to recover; (3) Most servers aren’t designed to handle multiple simultaneous long-lived connections. • The second technique, sometimes called service streaming, uses one or more long-lived XMLHttpRequest calls. In this technique, you can make the long-lived HTTP connection(s) anytime after the page is loaded and close and reopen the connection whenever you need, for example, to recover from a failed connection. The HTTP connection is kept open and data is pushed to the client from the server in the same manner as page streaming. An advantage to service streaming, however, is that you can push just about any data you want as long as you can process it in JavaScript (like JSON). With page streaming, you can only push HTML tags.

Service Streaming Communication

Web Remoting Pitfalls • The Web has been around long enough that users are accustomed to certain conventions. • You expect to get a visual progress indicator from the browser when waiting for a page to load • You expect to be able to click the back button and see the last Web page you were viewing • You expect to be able to bookmark pages so you can return directly to them at a later time • Ajax introduces forms of interaction that break these conventions.

Visual Feedback • One of the easiest problems to fix that Ajax introduces is a lack of visual feedback. With the traditional use of HTTP, when a user clicks on a hyperlink the browser provides some form of visual feedback that work is being done, like a spinning image or a progress bar. When you use the XMLHttpRequest object the browser does not provide any indication to the user that something is happening. • A simple fix for this is to display a text message or animated image while the XMLHttpRequest is being completed and hide the indicator when the request is complete. • Progress indicator examples:

Browsing History • As a user surfs the Web, the browser stores the URLs for all the pages the user has visited in a cache, commonly called the history. The browser then enables the Back and Forward buttons to allow the user to navigate back and forward through the browsing history. • When a request is made using XMLHttpRequest, the URL is not stored in the browser’s history. • You can avoid this pitfall by designing your application such that elements that look like hyperlinks are traditional hyperlinks that do a page reload and get entered into the browser’s history. Elements of your application that use XMLHttpRequest don’t have to fool the user into thinking they should behave like a traditional hyperlink. • JavaScript libraries like YUI and Dojo also have solutions.

Bookmarking • With static pages, the URL in your browser’s navigation bar can be bookmarked and you can return to that page by simply clicking on the bookmark. Therefore, users will expect this of your Ajax Web application as well. However, an XMLHttpRequest could be made and the response used to change a significant section of the page but the URL in the browser’s navigation bar will not change. • You can reduce your user’s expectations of bookmarking-ability by only changing content on the page that strictly has an “application” feel to it, but this will only take you so far. It’s important that you design for the ability to bookmark the state of your Web application even though you may have to provide a non-traditional way for the user to bookmark it (i.e., Google Maps “Link to this page” hyperlink). • JavaScript libraries like YUI and Dojo also provide solutions.

Same-Origin Policy • This policy prevents JavaScript code from reading or setting the properties of windows and documents that have a different origin than the document containing the script. The origin of a document includes the protocol, the domain name (host), and port of the URL from which the document was loaded. Documents belong to the same origin if and only if those three values are the same. • In other words, if the Web page in which your JavaScript code is loaded was pulled from http://www.ajaxbook.com and you wanted to use an XMLHttpRequest to directly grab some data from an Amazon Web service you would be out of luck. • The same-origin policy was implemented to prevent malicious scripts in one frame or window from accessing personal content in another frame or window.

The only current, cross-browser exception to the same-origin policy applies to windows and frames. The domain property of the Document object, by default, contains the hostname of the server from which the document was loaded. JavaScript in two different windows or frames can change their domain property to the same domain to interact with each other. However, the domain property can only be set to a valid domain suffix of the default value. For example, if the default value is “www.ajaxbook.com” then you can only change the value to “ajaxbook.com”. The domain set must have at least one dot in it so it cannot be set to a top-level domain, like “com”. • Another exception to the same-origin policy has been drafted by the W3C and implemented by Firefox 3, called W3C’s Access Control for Cross-site Requests.

Chapter 5 Web Remoting Techniques – the A in Ajax