eXtensible Markup Language (XML) & JavaScript Object Notation (JSON) in AJAX

eXtensible Markup Language (XML) & JavaScript Object Notation (JSON) in AJAX Chapter 12

The bad way to store data My note: BEGIN FROM: Alice Smith (alice@example.com) TO: Robert Jones (roberto@example.com) SUBJECT: Tomorrow's "Birthday Bash" event! MESSAGE (english): Hey Bob, Don't forget to call me this weekend! PRIVATE: true END • we could send a file like this from the server to browser with Ajax • what's wrong with this approach? • The data format is not standardized • It is unlikely to find any off-the-shelf tools to help break apart the data and process it

XML: A better way of storing data <?xml version="1.0" encoding="UTF-8"?> <note private="true"> <from>Alice Smith (alice@example.com)</from> <to>Robert Jones (roberto@example.com)</to> <subject>Tomorrow's "Birthday Bash" event!</subject> <message language="english"> Hey Bob, Don't forget to call me this weekend! </message> </note> • eXtensible Markup Language (XML) is a format for storing nested data with tags and attributes • lots of existing data on the web is stored in XML format

What is XML? • XML: a "skeleton" for creating markup languages • XML syntax is mostly identical to HTML's • you already know it! <elementattribute="value">content</element> • languages written in XML specify: • names of tags in HTML: h1, div, img, etc. • names of attributes in HTML: id/class, src, href, etc. • rules about how they go together in HTML: inline vs. block-level elements • used to present complex data in human-readable form • "self-describing data"

Anatomy of an XML file <?xml version="1.0" encoding="UTF-8"?>  <note private="true">  <from>Alice Smith (alice@example.com)</from> <to>Robert Jones (roberto@example.com)</to> <subject>Tomorrow's "Birthday Bash" event!</subject> <message language="english"> Hey Bob, Don't forget to call me this weekend! </message> </note> • begins with an <?xml ... ?> header tag • has a single root element (in this case, note) • tag, attribute, and comment syntax is just like HTML • XML tag names can be of any length and can contain letters, digits, underscores, hyphens and periods • Must begin with either a letter or an underscore, • should not begin with “xml” in any combination of uppercase and lowercase letters,

Uses of XML • XML is a portable, widely supported, open technology for data storage and exchange • XML data comes from many sources on the web: • web servers store data as XML files • databases sometimes return query results as XML • web services use XML to communicate • XML is the de facto universal format for exchange of data • XML languages are used for music, math, vector graphics • popular use: RSS for news feeds & podcasts • An XML parser is responsible for identifying components of XML documents (typically files with the .xml extension)

What tags are legal in XML? • when designing XML data, you choose how to best represent the data • large or complex pieces of data become tags • smaller details and metadata with simple types (integer, string, boolean) become attributes • examples: • an email message might use tags called to, from, subject • a library might use tags called book, title, author • rule of thumb: data = tag, metadata = attribute <?xml version = "1.0"?>  <player> <firstName>John</firstName> <lastName>Doe</lastName> <battingAverage>0.375</battingAverage> </player>

What tags are legal in XML? <?xml version = "1.0"?>  <article> <title>Simple XML</title> <date>July 4, 2007</date> <author> <firstName>John</firstName> <lastName>Doe</lastName> </author> <summary>XML is pretty easy.</summary> <content>This chapter presents examples that use XML.</content> </article>

XML and Ajax • web browsers can display XML files, but often you instead want to fetch one and analyze its data • the XML data is fetched, processed, and displayed using Ajax • (XML is the "X" in "Ajax") • It would be very clunky to examine a complex XML structure as just a giant string! • luckily, the browser can break apart (parse) XML data into a set of objects • there is an XML DOM, very similar to the HTML DOM

XML DOM tree structure <?xml version="1.0" encoding="UTF-8"?> <categories> <category>children</category> <category>computers</category> <category>cooking</category> <category>finance</category> </categories> • the XML tags have a tree structure • DOM nodes have parents, children, and siblings

Recall: Javascript XML (HTML) DOM • The DOM properties and methods* we already know can be used on XML nodes: • properties: • firstChild, lastChild, childNodes, nextSibling, previousSibling, parentNode • nodeName, nodeType, nodeValue, attributes • methods: • appendChild, insertBefore, removeChild, replaceChild • getElementsByTagName, getAttribute, hasAttributes, hasChildNodes • caution: cannot use HTML-specific properties like innerHTML in the XML DOM! * (though not Prototype's, such as up, down, ancestors, childElements, or siblings)

Common Node properties and methods

Recall: Javascript XML (HTML) DOM NodeListproperty and method. Element property and methods.

Navigating the node tree • caution: can only use standard DOM methods/properties in XML DOM (NOT Prototype's) • caution: can't use ids or classes to use to get specific nodes (no $ or $$). Instead: // returns all child tags inside node that use the given element var elms = node.getElementsByTagName("tagName"); • caution: can't use innerHTMLto get the text inside a node. Instead: vartext = node.firstChild.nodeValue; • caution: can't use .attributeNameto get an attribute's value from a node. Instead: varattrValue = node.getAttribute("attrName");

Using XML data in a web page • use Ajax to fetch data • use DOM methods to examine XML: • XMLnode.getElementsByTagName("tag") • extract the data we need from the XML: • XMLelement.getAttribute("name"), XMLelement.firstChild.nodeValue, etc. • create new HTML nodes and populate with extracted data: • document.createElement("tag"), HTMLelement.innerHTML • inject newly-created HTML nodes into page • HTMLelement.appendChild(element)

Fetching XML using AJAX (template) new Ajax.Request("url", { method: "get", onSuccess: functionName } ); ... function functionName(ajax) { do something with ajax.responseXML; } • ajax.responseText contains the XML data in plain text • ajax.responseXML is a pre-parsed XML DOM object

Fetching XML using AJAX (template) <?xml version="1.0" encoding="UTF-8"?> <employees> <lawyer money="5"/> <janitor name="Sue"><vacuumcleaner/></janitor> <janitor name="Bill">too poor</janitor> </employees> • We can use DOM properties and methods on ajax.responseXML: // zeroth element of array of length 1 varemployeesTag= ajax.responseXML.getElementsByTagName("employees")[0]; // how much money does the lawyer make? varlawyerTag = employeesTag.getElementsByTagName("lawyer")[0]; var salary = lawyerTag.getAttribute("money"); // "5" // array of 2 janitors varjanitorTags = employeesTag.getElementsByTagName("janitor"); var excuse = janitorTags[1].firstChild.nodeValue; // " too poor "

Analyzing a fetched XML file using DOM (2) <?xml version="1.0" encoding="UTF-8"?> <employees> <lawyer money="5"/> <janitor name="Sue"><vacuumcleaner/></janitor> <janitor name="Bill">too poor</janitor> </employees> • What are the results of the following expressions? // zeroth element of array of length 1 varemployeesTag = ajax.responseXML.getElementsByTagName("employees")[0]; • employeesTag.firstChild • ajax.responseXML.getElementsByTagName("lawyer") • employeesTag.getElementsByTagName("janitor").length • employeesTag.getElementsByTagName("janitor")[0].firstChild • employeesTag.getElementsByTagName("janitor")[1].firstChild

Larger XML file example <?xml version="1.0" encoding="UTF-8"?> <bookstore> <book category="cooking"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year><price>30.00</price> </book> <book category="computers"> <title lang="en">XQuery Kick Start</title> <author>James McGovern</author> <year>2003</year><price>49.99</price> </book> <book category="children"> <title lang="en">Harry Potter</title> <author>J K. Rowling</author> <year>2005</year><price>29.99</price> </book> <book category="computers"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year><price>39.95</price> </book> </bookstore>

Larger XML file example // make a paragraph for each book about computers var books = ajax.responseXML.getElementsByTagName("book"); for (vari = 0; i < books.length; i++) { var category = books[i].getAttribute("category"); if (category == "computers") { // extract data from XML var title = books[i].getElementsByTagName("title")[0].firstChild.nodeValue; var author = books[i].getElementsByTagName("author")[0].firstChild.nodeValue; // make an XHTML <p> tag containing data from XML var p = document.createElement("p"); p.innerHTML = title + ", by " + author; document.body.appendChild(p); } }

Example1: Body of HTML file <body> <input type = "radio" checked = "unchecked" name ="Books" value = "all" /> All Books <input type = "radio" checked = "unchecked" name = "Books" value = "simply"/> Simply Books <input type = "radio" checked = "unchecked" name = "Books" value = "howto" /> How to Program Books <input type = "radio" checked = "unchecked" name = "Books" value = "dotnet" /> .NET Books <input type = "radio" checked = "unchecked" name = "Books" value = "javaccpp" /> Java, C, C++ Books <input type = "radio" checked = "checked" name = "Books" value = "none" /> None <br/> <div id = "covers"></div> </body> • What are the results of the following expressions?

Example1 : Sample XML file at the server <?xml version = "1.0"?> <covers> <baseurl>http://test.deitel.com/examples/iw3htp4/flex/coverViewer/thumbs/</baseurl> <cover> <image>simplycpp.jpg</image> <title>Simply C++</title> </cover> <cover> <image>simplyvb2005.jpg</image> <title>Simply VB 2005</title> </cover> <cover> <image>simplyjava.jpg</image> <title>Simply Java</title> </cover> </covers>

Example1 : JS file window.onload = function() { varmyimg= document.getElementsByName("Books"); for (vari= 0; i < myimg.length; i++) { if (i == myimg.length -1) myimg[i].onclick = clearTable; else myimg[i].onclick = getImages; } }; // set up and send the asynchronous request to the XML file function getImages() { varurl = this.value + ".xml"; // attempt to create the XMLHttpRequest and make the request new Ajax.Request( url, { method: "get", onSuccess: processResponse, } ); } // end function getImages // deletes the data in the table. function clearTable() { document.getElementById( "covers" ).innerHTML = ''; }// end function clearTable

Example1 : JS file // parses the XML response; dynamically creates a table using DOM and // populates it with the response data; displays the table on the page function processResponse(asyncRequest) { clearTable(); // prepare to display a new set of images // get the covers from the responseXML varcovers = asyncRequest.responseXML.getElementsByTagName("cover" ); // get base URL for the images varbaseUrl = asyncRequest.responseXML.getElementsByTagName("baseurl" ).item(0).firstChild.nodeValue; // get the placeholder div element named covers var output = document.getElementById( "covers" ); // create a table to display the images varimageTable = document.createElement( 'table' ); // create the table's body vartableBody = document.createElement( 'tbody' ); varrowCount = 0; // tracks number of images in current row varimageRow = document.createElement( "tr" ); // create row <?xml version = "1.0"?> <covers> <baseurl>http://test.deitel.com/examples/iw3htp4/flex/coverViewer/thumbs/</baseurl> <cover> <image>simplycpp.jpg</image> <title>Simply C++</title> </cover> <cover> <image>simplyvb2005.jpg</image> <title>Simply VB 2005</title> </cover> <cover> <image>simplyjava.jpg</image> <title>Simply Java</title> </cover> </covers>

Example1 : JS file // place images in row for ( vari = 0; i < covers.length; i++ ) { var cover = covers.item( i ); // get a cover from covers array // get the image filename var image = cover.getElementsByTagName( "image" )[0].firstChild.nodeValue; // create table cell and img element to display the image varimageCell = document.createElement( "td" ); varimageTag = document.createElement( "img" ); // set img element's src attribute imageTag.setAttribute( "src", baseUrl + image ); imageCell.appendChild( imageTag ); // place img in cell imageRow.appendChild( imageCell ); // place cell in row rowCount++; // increment number of images in row // if there are 6 images in the row, append the row to table and start a new row if ( rowCount == 6 && i + 1 < covers.length) { tableBody.appendChild( imageRow ); imageRow = document.createElement( "tr" ); rowCount = 0; } // end if statement } // end for statement tableBody.appendChild( imageRow ); // append row to table body imageTable.appendChild( tableBody ); // append body to table output.appendChild( imageTable ); // append table to covers div } // end function processResponse <?xml version = "1.0"?> <covers> <baseurl>http://test.deitel.com/examples/iw3htp4/flex/coverViewer/thumbs/</baseurl> <cover> <image>simplycpp.jpg</image> <title>Simply C++</title> </cover> <cover> <image>simplyvb2005.jpg</image> <title>Simply VB 2005</title> </cover> <cover> <image>simplyjava.jpg</image> <title>Simply Java</title> </cover> </covers>

Example2 : book search Freakonomics|Steven Levitt|finance|2006|18.00 Harry Potter and the Sorceror'sStone|J.K. Rowling|children|1998|19.99 Breakfast for Dinner|Amanda Camp|cooking|2009|22.00 Web Programming Step-by-Step|MartyStepp and Jessica Miller|computers|2009|999.99 Building Java Programs|StuartReges and Marty Stepp|computers|2007|90.00 Core Java, 8th edition|Cay Horstmann|computers|2007|35.00 Personal Finance for Dummies|Eric Tyson|finance|2006|14.95 21 Burgers for the 21st Century|Stuart Reges|cooking|2010|75.00 ... books.txt Part of book-search.html <body> <div id="categoryarea"> <p>Categories:</p> <ul id="categories"> <li>Fetching...</li> </ul> </div> <div id="bookarea"> <p>Books for sale:</p> <ul id="books"> <li>Choose a category...</li> </ul> </div> </body> Part of book-search.js executed when the document is loaded window.onload = function() { new Ajax.Request("books1.php", { method: "GET", onSuccess: showCategories, onFailure: ajaxFailed onException: ajaxFailed } ); };

Example2 : book search Freakonomics|Steven Levitt|finance|2006|18.00 Harry Potter and the Sorceror'sStone|J.K. Rowling|children|1998|19.99 Breakfast for Dinner|Amanda Camp|cooking|2009|22.00 Web Programming Step-by-Step|MartyStepp and Jessica Miller|computers|2009|999.99 ... $BOOKS_FILE = "books.txt"; header("Content-type: application/xml"); print "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"; print "<categories>\n"; $categories = array(); foreach (file($BOOKS_FILE) as $line) { list($title, $author, $book_category, $year, $price) = explode("|", trim($line)); if (!isset($categories[$book_category])) { $categories[$book_category] = 1; } } ksort($categories); foreach ($categories as $book_category => $useless) { print "\t<category>$book_category</category>\n"; } print "</categories>\n"; Part of books1.php that generate xml file of book categories <categories> <category>children</category> <category>computers</category> <category>cooking</category> <category>finance</category> </categories> Generated xml categories

Example2 : book search Part of book-search.js that processes the XML file function showCategories(ajax) { // clear out the list of categories while ($("categories").firstChild) { $("categories").removeChild($("categories").firstChild); } // add all categories from the XML to the page's bulleted list var categories = ajax.responseXML.getElementsByTagName("category"); for (vari = 0; i < categories.length; i++) { varcategoryName = categories[i].firstChild.nodeValue; // create a new <li> tag and add it to the page varli = document.createElement("li"); li.innerHTML = categoryName; li.onclick = categoryClick; $("categories").appendChild(li); } } <categories> <category>children</category> <category>computers</category> <category>cooking</category> <category>finance</category> </categories> Generated xml categories

Example2 : book search Part of book-search.js that shows what ajax request is fired when clicking on a category function categoryClick() { new Ajax.Request("books1.php", { method: "GET", parameters: {category: this.innerHTML}, onSuccess: showBooks, onFailure: ajaxFailed, onException: ajaxFailed } ); }

Example2 : book search Freakonomics|Steven Levitt|finance|2006|18.00 Harry Potter and the Sorceror'sStone|J.K. Rowling|children|1998|19.99 ... $BOOKS_FILE = "books.txt"; # Removes all characters except letters/numbers from query parameters function filter_chars($str) { return preg_replace("/[^A-Za-z0-9_]*/", "", $str); } $category = ""; if (isset($_REQUEST["category"])) { $category = filter_chars($_REQUEST["category"]); } # the below will be executed if ($category) print "<books>\n"; foreach (file($BOOKS_FILE) as $line) { list($title, $author, $book_category, $year, $price) = explode("|", trim($line)); if ($book_category == $category) { # print it as XML print "\t<book category=\"$category\" year=\"$year\" price=\"$price\">\n"; print "\t\t<title>$title</title>\n"; print "\t\t<author>$author</author>\n"; print "\t</book>\n"; } } print "</books>\n"; Part of books1.php that generate xml file of a certain category <books> <book category="children" year="1998" price="19.99"> <title>Harry Potter and the Sorceror's Stone</title> <author>J.K. Rowling</author> </book> <book category="children" year="1961" price="8.99"> <title>Go, Dog, Go!</title> <author>P.D. Eastman</author> </book> </books>

Example2 : book search function showBooks(ajax) { // clear out the list of categories while ($("books").firstChild) { $("books").removeChild($("books").firstChild); } // add all books from the XML to the page's bulleted list var books = ajax.responseXML.getElementsByTagName("book"); for (vari = 0; i < books.length; i++) { vartitleNode = books[i].getElementsByTagName("title")[0]; varauthorNode = books[i].getElementsByTagName("author")[0]; var title = titleNode.firstChild.nodeValue; var author = authorNode.firstChild.nodeValue; var year = books[i].getAttribute("year"); varli = document.createElement("li"); li.innerHTML = title + ", by " + author + " (" + year + ")"; $("books").appendChild(li); } } Part of books1.js that shows how client add ajax results (xml file of a certain category ) to page <books> <book category="children" year="1998" price="19.99"> <title>Harry Potter and the Sorceror's Stone</title> <author>J.K. Rowling</author> </book> <book category="children" year="1961" price="8.99"> <title>Go, Dog, Go!</title> <author>P.D. Eastman</author> </book> </books>

DTD versus Well-formed 1. Well-formedXML allows you to invent your own tags. 2. Valid XML involves a DTD (Document Type Definition): "rule books" describing which tags/attributes you want to allow in your data • used to validate XML files to make sure they follow the rules of that "flavor" • the W3C HTML validator uses an HTML schema to validate your HTML (related to <!DOCTYPE html> tag) • "rule books"are optional; if you don't have one, there are no rules beyond having well-formed XML syntax • for more info: • W3C XML Schema • Document Type Definition (DTD) ("doctype")

Pros and cons of XML • pro: • standard open format; don't have to "reinvent the wheel" for storing new types of data • can represent almost any general kind of data (record, list, tree) • easy to read (for humans and computers) • lots of tools exist for working with XML in many languages • con: • bulky syntax/structure makes files large; can decrease performance (example) • can be hard to "shoehorn" data into a good XML format • JavaScript code to navigate the XML DOM is bulky and generally not fun

An example of XML data <?xml version="1.0" encoding="UTF-8"?> <note private="true"> <from>Alice Smith (alice@example.com)</from> <to>Robert Jones (roberto@example.com)</to> <to>Charles Dod (cdod@example.com) </to> <subject>Tomorrow's "Birthday Bash" event!</subject> <message language="english"> Hey Bob, Don't forget to call me this weekend! </message> </note> • fairly simple to read and understand • can be parsed by JavaScript code using XML DOM • Is there any other data format that is more natural for JS code to process?

JavaScript Object Notation (JSON) • JavaScript Object Notation (JSON): Data format that represents data as a set of JavaScript objects • invented by JS guru Douglas Crockford of Yahoo! • natively supported by all modern browsers (and libraries to support it in old ones) • not yet as popular as XML, but steadily rising due to its simplicity and ease of use

Background: Creating a new object var name = { fieldName: value, ... fieldName: value }; • in JavaScript, you can create a new object without creating a class • you can add properties to any object even after it is created (z) varpt = { x: 4, y: 3 }; pt.z = -1; alert("(" + pt.x + ", " + pt.y + ", " + pt.z + ")"); // (4, 3, -1)

More about JavaScript object syntax var person = { name: "Philip J. Fry", // string age: 23, // number "weight": 172.5, // number friends: ["Farnsworth", "Hermes", "Zoidberg"], // array getBeloved: function() { return this.name + " loves Leela"; } }; alert(person.age); // 23 alert(person["weight"]); // 172.5 alert(person.friends[2])); // Zoidberg alert(person.getBeloved()); // Philip J. Fry loves Leela • an object can have methods (function properties) that refer to itself as this • can refer to the fields with .fieldName or ["fieldName"] syntax • field names can optionally be put in quotes (e.g. weight above)

Expressing XML message data as a JavaScript object <?xml version="1.0" encoding="UTF-8"?> <note private="true"> <from>Alice Smith (alice@example.com)</from> <to>Robert Jones (roberto@example.com)</to> <to>Charles Dod (cdod@example.com) </to> <subject>Tomorrow's "Birthday Bash" event!</subject> <message language="english"> Hey Bob, Don't forget to call me this weekend! </message> </note> • Each attributeand tagcould become a property or sub-object within the overall message object { "private": "true", "from": "Alice Smith (alice@example.com)", "to": [ "Robert Jones (roberto@example.com)", "Charles Dodd (cdodd@example.com)" ], "subject": "Tomorrow's \"Birthday Bash\" event!", …

The equivalant JSON data <?xml version="1.0" encoding="UTF-8"?> <note private="true"> <from>Alice Smith (alice@example.com)</from> <to>Robert Jones (roberto@example.com)</to> <to>Charles Dod (cdod@example.com) </to> <subject>Tomorrow's "Birthday Bash" event!</subject> <message language="english"> Hey Bob, Don't forget to call me this weekend! </message> </note> { "private": "true", "from": "Alice Smith (alice@example.com)", "to": [ "Robert Jones (roberto@example.com)", "Charles Dodd (cdodd@example.com)" ], "subject": "Tomorrow's \"Birthday Bash\" event!", "message": { "language": "english", "text": "Hey Bob, Don't forget to call me this weekend!" } }

Valid JSON var student = { // no variable assignment "first_name": 'Bart', // strings must be double-quoted last_name: "Simpson", // property names must be quoted "birthdate": new Date("April 1, 1983"), // Date objects not supported "enroll": function() { // Functions not supported this.enrolled = true; } }; • JSON has a few rules that differ from regular JS: • Strings must be quoted (in JS, single- or double-quoted are allowed) • All property/field names must be quoted • Unsupported types: Function, Date, RegExp, Error • All others supported: Number, String, Boolean, Array, Object, null • Numerous validators/formatters available: JSONLint, JSON Formatter & Validator, Free Formatter, JSON Validator

Browser JSON methods • you can use Ajax to fetch data that is in JSON format • then call JSON.parse on it to convert it into an object • then interact with that object as you would with any other JavaScript object

JSON expressions exercise var data = JSON.parse(this.responseText); { "window": { "title": "Sample Widget", "width": 500, "height": 500 }, "image": { "src": "images/logo.png", "coords": [250, 150, 350, 400], "alignment": "center" }, "messages": [ {"text": "Save", "offset": [10, 20]}, {"text": "Help", "offset": [ 0, 50]}, {"text": "Quit", "offset": [30, 15]} ], "debug": "true" } Given the JSON data at right, what expressions would produce: • The window's title? • The image's third coordinate? • The number of messages? • The y-offset of the last message? var title = data.window.title; varcoord = data.image.coords[2]; varlen = data.messages.length; var y = data.messages[len - 1].offset[1];

JSON example: Books Suppose we have a service books_json.php about library books. • If no query parameters are passed, it outputs a list of book categories: { "categories": ["computers", "cooking", "finance", ...] } Supply a category query parameter to see all books in one category: http://YOURWEBSERVER/books_json.php?category=cooking { "books": [ {"category": "cooking", "year": 2009, "price": 22.00, "title": "Breakfast for Dinner", "author": "Amanda Camp"}, {"category": "cooking", "year": 2010, "price": 75.00, "title": "21 Burgers for the 21st Century", "author": "Stuart Reges"}, ... ] }

JSON exercise • Write a page that processes this JSON book data. • Initially the page lets the user choose a category, created from the JSON data. After choosing a category, the list of books in it appears: (example below after choosing finance category)

Working with JSON book data function showBooks() { // add all books from the JSON data to the page's bulleted list var data = JSON.parse(this.responseText); for (vari = 0; i < data.books.length; i++) { var li = document.createElement("li"); li.innerHTML = data.books[i].title + ", by " + data.books[i].author + " (" + data.books[i].year + ")"; document.getElementById("books").appendChild(li); } }

Bad style: the eval function // var data = JSON.parse(this.responseText); var data = eval(this.responseText); // don't do this! ... • JavaScript includes an eval keyword that takes a string and runs it as code • this is essentially the same as what JSON.parse does, • but JSON.parse filters out potentially dangerous code; eval doesn't • eval is evil and should not be used!

eXtensible Markup Language (XML) & JavaScript Object Notation (JSON) in AJAX