Introduction to XQuery and eXist

Introduction to XQuery and eXist Week 21 DSA

Refresher on XPath • XML databases • XQuery Applications • Whiskies (again) • A simple blog • The XQuery Wikibook DSA - XQuery

XPath (1) • Hierarchical file systems have been navigable with path expression since Unix • /abc/cde/../../efg i.e. ? • Problem • Only child, parent and root can be accessed in one step • Only one node addressable DSA - XQuery

XPath (2) • Solution – extend the path language • select multiple items • /abc/cde/../../efg/* - all children of the node /efg • //x - all x nodes anywhere in the tree • select a subset of items by appending predicate (filter) – nodes selected if predicate evaluates to true • //x[@size>1000] • functions in predicate • //x[ends-with(@name,’.jpeg’)] • multiple filters • //x[@owner=‘fred’]//images[ends-with(@name,’.jpeg’)] • select items by position • /abc[2] short for /abc[position() = 2] DSA - XQuery

XPath (3) • Core data type in XPath 2.0 is a sequence of items • Items are atomic values or XML elements • Sequence is flat • deep-equal('fred',('fred')) is true • deep-equal((‘fred’, ‘fred’), (‘fred’)) is false • deep-equal(('fred', ('bill','joe')),('fred', 'bill','joe')) is true • () is the empty sequence • count((‘fred’,() )) is 1 • Equality between sequences defined as non-empty intersection • (‘fred’,’joe’) = ‘joe’ is true • (‘fred’,’joe’) = (‘joe’,’bill’) is true • (‘fred’,’joe’) eq ‘joe’ is false DSA - XQuery

XQuery . • Problem • Need to be able to construct a tree as well as select nodes in an existing tree • Need for more complex searches a la SQL • Solution 1 – XML addition to existing languages • PHP + Simple XML • XML structures not compatible with language • Solution 2 – XSLT • Push (declarative, pattern-matching) or pull processing • XML- based • Two versions • 1.0 most common, in Browsers, Xalan • 2.0 e.g. Saxon • Solution 2 – XQuery • Functional • Pull only • Non XML • XQuery 1.0 and XSLT 2.0 share the same XPath 2.0 data model and function library DSA - XQuery

eXist Native XML Database • Open source • Wolfgang Meier is the chief architect • Written in Java • Deployable in different ways • Embedded in a Java application • Part of a Cocoon pipeline • As web application in Apache/Tomcat • With embedded Jetty HTTPserver (as on stocks) • Multiple Interfaces • REST – to Java servlet • SOAP • XML:RPC DSA - XQuery

Native XML database • Well-formed XML documents can be added to the database • They are stored in an efficient, searchable B+ tree structure • Documents (files) are organised into collections in a filestore • Non-XML resources (XQuery, CSS, JPEG ..), etc can be stored as binary DSA - XQuery

Whisky example • A base XML file • Some XQuery Scripts • Simple list • Another simple list • Select Whiskies • Select with a Form • Generate a kml map DSA - XQuery

Simple Xquery – list1.xql for $d in //Distillery return <div> <h2>{$d/Name}</h2> <p>{$d/WhiskyDescription}</p> </div> DSA - XQuery

Executing an XQuery eXist: Server Client Browser User clicks link Get a.xql parameters servlet fetch a.xql parameters a.xql eXist DB html XQuery Engine render DSA - XQuery

Simple XQuery – list2.xql declare option exist:serialize "method=xhtml media-type=text/html"; <table border='2'> <tr> <th>Name</th><th>Address</th> </tr> {for $w in //Distillery return <tr> <td>{data($w/Name)}</td> <td>{data($w/Address)}</td> </tr> } </table> DSA - XQuery

XQuery explained XQuery ‘variable’ Setting the output mime type declare option exist:serialize "method=xhtml media-type=text/html"; <table border='2'> <tr> <th>Name</th><th>Address</th> </tr> { for $w in //Distillery return <tr> <td>{data($w/Name)}</td> <td>{data($w/Address)}</td> </tr> } </table> XQuery inside XML XPath expression to select nodes Another XPath expression Function to get the text DSA - XQuery

XQuery .. • Nesting XML and XQuery • XML to XQuery • { … } • XQuery to XML <tag> … </tag> • Must be well-formed XML – single root - <div> and <span> handy • Constructed structures • XML element <Position latitude=’51’ longitude=‘-2.5’ /> or element Position { attribute latitude ’51’, attribute longitude ‘2.5’ } • Sequence • ('fred', <tag a='4'/>, 5) DSA - XQuery

XQuery .. • control constructs are expressions and hence composable • if (cond) then exp else exp • for ..let.. where .. order by .. return – FLWOR • function call • variable value ($d) • ‘Atomisation’ sometimes needed in output • Element to characters • $d/Name • <Name>Glenfiddich</Name> • data($d/Name) • Glenfiddich DSA - XQuery

XQuery … • Functions and Modules • Typed arguments and return • Recursion • XPath functions • eXist functions • Database management • HTTP interface • .. • Functional language • let $x := 5 binds the value 5 to the variable $x • Can’t write iterative code – what’s the output? • let $y := 1 • return • for $x in (1 to 5) • let $y := $y * 2 • return $y DSA - XQuery

XQuery FLWOR expression • FLWOR for $x in sequence let $a := expression, $b := expression where condition order by $var return expression • Returns a Sequence of nodes • Compare with SQL select columns from tables where condition order by • Returns a Relation (table) DSA - XQuery

Search Query • Enter a name or part name • Match against the Distillery name • List the matches DSA - XQuery

List 3 FLWOR expression let $name := request:request-parameter("name","") return <table border='2'> <tr><th>Name</th><th>Address</th></tr> {for $d in //Distillery[contains(Name,$name)] return <tr><td>{data($d/Name)}</td> <td>{data($d/Address)}</td> </tr> } </table> eXist function DSA - XQuery

An XQuery sticky form • The script here has several shifts between XML to XQuery • Whole interface in one script • Equivalent to PHP + MySQL. DSA - XQuery

let $name := request:request-parameter("name","") return <html> <form method="get" action=""> <label>Enter Name or part of Name</label> <input type="text" name="name" size="10"/> </form> <table border='2'> <tr><th>Name</th><th>Address</th></tr> { for $d in //Distillery[contains(Name,$name)] return <tr> <td>{data($d/Name)}</td> <td>{data($d/Address)}</td> </tr> } </table> </html> list4.xql DSA - XQuery

Generate a kml overlay • Output is kml • Need to specify the mime type so that the browser will link to Google Earth (if set up correctly) • Or create a Google Map link: • http://maps.google.com/maps?q=http://www.cems.uwe.ac.uk/xmlwiki/whisky/map.xql DSA - XQuery

Kml declare option exist:serialize "method=xhtml indent=yes media-type=application/vnd.google-earth.kml+xml"; <Document> <name>Distilleries of Scotland 2</name> {for $d in //Distillery return <Placemark> <name>{data($d/Name)}</name> <description>{data($d/WhiskyDescription)}</description> <Point> <coordinates> {data($d/Position/@longitude)}, {data($d/Position/@latitude)},0 </coordinates> </Point> </Placemark> } </Document> distkml.xql DSA - XQuery

Reading an API • Brooklyn Museum API • Simple search • Returning XML • Returning HTML DSA - XQuery

Source let $uri := "http://www.brooklynmuseum.org/opencollection/api/"let $apikey := "yPbZKsKZLC"let $keyword:= request:get-parameter("keyword","Anubis")let $request := concat($uri,"?api_key=",$apikey,"&version=1&include_item_caption=true&require_image=true&method=collection.search&keyword=",$keyword)return doc($request) DSA - XQuery

Uploading DSA CW2 metadata • upload.xql DSA - XQuery

FOLD and StudentsOnline • StudentsOnline DSA - XQuery

XQuery Wikibook • A large number of worked examples • http://en.wikibooks.org/wiki/XQuery DSA - XQuery

Trying eXist • Demo site • http://exist-db.org/ • Local installation • Simple install from the exist-db site • Use the Java client to • load files • move, copy, rename files • edit files in situ (but no Save-as) • execute queries • backup DSA - XQuery

Other Native XML databases • MarkLogic • MarkMail • Sedna • Berkleydb • Many Relational databases include partial support for XML DSA - XQuery

Introduction to XQuery and eXist