1 / 24

Data, schemas and Applications

Delve into data structures, schemas, and applications with a focus on XML, SQL, and Yahoo Pipes. Understand the value of combining data from various sources for informed decision-making.

annmaries
Download Presentation

Data, schemas and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data, schemas and Applications UFIEKG-20-2 Chris Wallace Praminda Caleb-Solly

  2. Assessment • 50% exam • 40% multiple choice • 2 *30% design questions • 50% coursework • Term 1 – well-defined group project + individual component • using Yahoo Pipes and XML with SQL • Term 2 – open individual project • XML and XQuery

  3. Resources • UWE Online • Schedule • Web sites • Week by Week material • DSA2007 blog • http://dsa2007.blogspot.com/ • Informal news and commentary • please add your own comments • del.icio.us • e.g. http://del.icio.us/morelysq/pipes • Social bookmarking site

  4. Books • Use free resources on the web • Wikipedia • Wiki Books • A number of useful books on XML for a tenner. • XML - Learning by Example - by Robert Mellar, reasonably priced at 10.49 • There's another book for a tenner XML in Easy Stages by Mike McGrath. • At the same price there is a SAMS book • Reference book • XML in a Nutshell - Eliottte Rusty Harold

  5. Themes • Data (and Information) • Schemas • Tools and Languages • Applications • Learning to learn

  6. Data • The world is awash with data • Cheap memory means it’s possible to keep every transaction, every document • The Web provides an infrastructure for individuals and organisations to make huge volumes of data available • Flickr • Wikipedia • Facebook • BBC

  7. Information • Data is useful for informing decisions, especially if it can be combined: • Price comparision • Nearby resources • Informed public opinion • But • Data is hard to combine • Data is of variable and often unknown quality and source (who, when, where, why) • The web as distributed datasets • The semantic web

  8. Schemas • How data is structured • CSV (Comma separated variable) • EXIF • Relational Tables • Tagged (HTML, XML) • How we can check that data is in the right format (Validation) • How we can describe the rules of valid data.

  9. Tools • A new language • RSS • A way of disseminating News • A new development tool • Yahoo Pipes • A Visual programming language for ‘Mashups’ • In the future • XML, SQL, XQuery, a bit of PHP ...

  10. Applications • Combining data from multiple sources to increase the value of the data • Developing our own data sources • Seaching and visualizing complex data

  11. Learning to learn • Developing skills in learning new tools and languages • New langauges and tools appear constantly • Very exciting • Very time-consuming • Very confusing (should I bother?) • The more you learn, the quicker it gets

  12. Learning Objectives for the next few weeks • Data • News and weather • Schema • RSS (and XML) • Tools • Yahoo Pipes • Application • A personal news feed

  13. Spreading the news • Problem: • I want to disseminate news about my project/company, and allow interested people to read it. • e.g. the university wants to spread the news about successful staff • Solution • publish a page of news on the website in HTML • Problems • how do visitors know when its changed? • news from different universities cannot be easily combined – (why?)

  14. Extend HTML • Solution • UWE makes up its own set of additional tags to add to the HTML page • <newsitem date=“2007-10-2”>…</newsitem> • Problem • A reader has to be able to understand this format – and many others!

  15. RSS • Solution • Standardise on one (or 4!) general formats of news feed, so that any reader can understand it • RSS • Really Simple Syndication • Rich Site Summary • Also ATOM • Characteristics • Basic content are items in reverse chronological order • Some basic tags such as links back to the full story.

  16. RSS at UWE • The UWE site http://www.uwe.ac.uk • Shows that there is an RSS feed • Which is this page http://info.uwe.ac.uk/news/uwenews/downloadxml.asp • This page appears to be a normal HTML page but underneath • View Source

  17. <?xml version="1.0" encoding="iso-8859-1"?> <rss version="2.0"> <channel> <title>UWE News</title> <link>http://www.uwe.ac.uk</link> <description>Latest UWE press releases</description> <image> <url>http://info.uwe.ac.uk/common/assets/2004Design/logo.gif</url> <title>University of the West of England</title> <link>http://www.uwe.ac.uk</link> </image> <pubDate>Sun, 1 Oct 2007 12:06:10 GMT</pubDate> <item> <title>UWE researcher wins funding for diabetes project</title> <link>http://info.uwe.ac.uk/news/uwenews/article.asp?item=1123</link> <description>What prevents insulin from being secreted naturally from the pancreatic beta cells of people who have diabetes? This question is key to research undertaken by Dr Aniko Varadi at the University of the...</description> </item> <item>…. </channel> </rss>

  18. RSS Feed Readers • Problem • How to keep up with changes to multiple RSS feeds • Solution • Application needed which • Is stateful – remembers what items you have read • Integrates multiple feeds into one ‘magazine’ • Polls RSS providers on a regular basis • Feed integrators Bloglines, Google Reader, reduce the load on the provider and provide some filtering • There is an RSS reader integrated into MyUWE

  19. RSS Mashup • Problem • I want to create my own feed which brings together and reorganises data from several RSS feeds, web sites, search engines • A MASHUP • Solution • Write your own script (later in XQuery) • Use Yahoo Pipes to do the job

  20. Yahoo Pipes • A visual programming environment • Allows standard components (modules) to be combined with data flows to produce a new combined component (a Pipe) • ‘Fill in the fields’ to configuring each module • Connect the modules with data flows • Look at the flow in any pipe

  21. How do you learn a new language or tool? • What approach works best for you? • What sources of information do you use? • How do you record your understanding as it develops?

  22. Pyramiding • 5 minutes – think about these questions on your own • 10 minutes – in pairs, look for differences and explore these • 10 minutes - groups

  23. Tutorials • Week 1 • Background Questionnaire • Revision • Preparation for next week • Watch the Google Tech talk on Yahoo Pipes • Find and subscribe to one RSS feed • Week 2 • Workshop on Yahoo Pipes

  24. Next week • Where do RSS feeds come from? • How does a RSS reader work? • How to roll your own RSS feed • Using Yahoo Pipes

More Related