Blogging, Googling, Syndication: What’s the Web coming to?

Blogging, Googling, Syndication: What’s the Web coming to? Ian Graham Enterprise IT Strategy & BMO Connect, T&S T: 416.513.5656 E: ian.graham@bmo.com W: http://www.utoronto.ca/ian/talks/

What this talk is about • The culture of web applications • What are the new applications about? • How are they ‘designed’ and put together • Where are they taking the ‘public’ Web • Examples (Lots) • That help to illustrate the above • Some conclusions What the talk is not about • Banking/financial services, software development, Web Services (tech details)

Location Connectivity Discovery Communication & Presence Community Blogging Syndication  RSS Google Jabber, ICQ Ebay, Yahoo, Slashdot, Amazon, Blogspots, … Web culture The right side lists just a few examples: there are many others …

A) Having a home: Blogging • Short for ‘Weblogging’ • A ‘log’ authored on / accessible over, the Web • Simple Web-site generation • Personal (or group) publishing & communications: • Users often publish regularly (daily, or weekly), and link to friends / colleagues / coworkers • Links establish virtual ‘communities’ with shared interests • Categorization for topics, etc. • Software is often free to download (and play with) • Free (or near) hosting available (see list at end) • Software often supports user-level scripting to building custom functions, behaviors

Some Example Blogs • http://www.nerdgirl.com • A ‘normal’ person (purely social site) • http://weblogs.mozillazine.org/roadmap/ [1] • Inventor of JavaScript – roadmap for development team • http://blogs.msdn.com/dmassy/archive/2004/06/16/157263.aspx • Internet Explorer development team member • http://www.joelonsoftware.com/ • Articles on software development • http://dear_raed.blogspot.com/2003_02_01_dear_raed_archive.html [2] • Reported from inside Iraq before and during last year’s war

Uses of Weblogs • Pure social networking • Post musings and share them with friends • (Semi-) professional information sharing • Post information for more general communication (e.g. news, current affairs, professional commentary) • Work-related networking • Post status reports, decisions and analyses, etc, for sharing / cross-linking with others. • Popularity? • There are millions of blogsites!

Blogspace • The collection of blog sites all across the Internet • Unconnected except for hand-created links in the pages

B) Two issues: • Letting people know what you have • Publishing information ‘about’ blog postings • RSS, Atom: XML languages for summary publishing • RSS  Resource Site Summary; RDF Site Summary; Really Simple Syndication …) – [ example later ] • Also used by regular web sites, news sites, etc. • Finding interesting/relevant stuff • Aggregation sites and clients • aggregating info ‘about’ published blogs / other sites • Headliner clients, topic-specific Web sites • Categorization and rating systems • Categorize blog items, Web page/sites • separate wheat from chaff

<item> <dc:creator>Dave Massy [MSFT] </dc:creator> <title>Security, Security, Security</title> <link> http://blogs.msdn.com/dmassy/archive/ 2004/06/20/160649.aspx </link> <pubDate>Sun, 20 Jun 2004 12:56:00 GMT </pubDate> <description> <P>This topic seems to be coming up <P> </P><img src ="http://blogs.msdn.com/ dmassy/aggbug/160649.aspx" width = "1" height = "1" /> …more text … </description> </item> E.g.: http://blogs.msdn.com/dmassy/  RSS link [3] B.1 RSS Feeds from a blogsite Time-stamped ‘summary’ about a blog item Metadata (for cataloging, indexing) Headline text A link back to the original … An ‘ad’ for the original content.

RSS RSS RSS or HTML RSS RSS B.2. Aggregators & Syndicate Retrieves and categorizes RSS from blog (and other) sites And provides a categorized feed to users, or a Web listing: • http://aggregator.weblogs.co.uk/ (a Blog aggregator) • http://meerkat.oreilly.com (an RSS feed aggregator) [4] • http://www.newzcrawler.com (an RSS desktop client) [5]

RSS feed of items in profilecategory 1065 Same, but use RSS 1.0 format for feed Aggregator APIs: • APIs for talking to aggregation sites • Simple, easy to use, often URL-based • E.g. Meerkat API -- • http://www.oreillynet.com/meerkat/ [6]http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html -- \ API documentation) • API based on arguments in the URL (docs above explain how it works). Here are some examples: • http://www.oreillynet.com/meerkat/?p=1065&_fl=rss [6.1,6.2] • http://www.oreillynet.com/meerkat/?p=1065&_fl=rss10

And Embeddable in Pages: ( . . . demo.html) [6.3] <html> <head> <title> Example of inclusion of RSS feeds using JavaScript </title> </head> <body> <h1> Heading For the Example </h1> <p> Here is some regular HTML -- ok, i didn't have time to make it pretty </p> <p> And here is the JavaScript example code: <script language="JavaScript" src="http://meerkat.oreillynet.com/?_fl=js"> </script> </body> </html> Returned script sent by Meerkat; writes out HTML summaries into file

Summary • Blogging is for individuals • RSS (and syndication) help build connectivity, and communities • Aggregators support simple APIs so it’s easy to build functions that integrate these tools with others (blogsites, portals, other..) • Richer communities via richer functionality • No conscious high-level architecture: people just try new ideas, and seeing what works (or doesn’t) • A heterogenous collection of apps, sharing using simple protocols, APIs. • Each application and protocol is simple (limited scope, easy to use ), and open (encourages reuse, extension) URL as a reference / API is key to this model

C. Googling • A web search engine • http://www.google.com [7] • Other stuff: • Blogger.com; Google groups, special search services, maps, Gmail (email) etc. • Some numbers: • Estimated space > 2,000,000 GB • Number of servers  100,000 (Linux) • Access to the index: • Via several APIs, some simple, and some complex

Google as a service • Via APIs that let you reuse the google engine as part of your own application: • A URL-based API (like Meerkat) • A full-blown Web services interface • The latter lets you, essentially, use Google as the search engine / database within your own applications (subject to restrictions)

Google’s URL-based API ( . . . demo-google.html) [7.1] <form id="search" method="get" action="http://www.google.com/custom"> <input type="hidden" name="cof" value="LW:600;LH:58;L:http://www.bmo.com/images/ ...banner.gif;GIMP:#cc0000;T:black; ...;AWFID:9262c37cefe23a86;"> <input type="hidden" name="domains" value=“bmo.com"> <input type="hidden" name="sitesearch" value=“bmo.com"> <input type="text" id="q" name="q" accesskey="s" size="30"> <input type="submit" id="submit" value="go"> </form> ----- Info ----------------------- http://www.google.com/faq_freewebsearch.html http://cyberzoide.developpez.com/html/google/

Google’s Web Services API • Experimental interface, but widely used • Requires registration for access key (simple, and free) • Provides library and tools for building simple applications, using SOAP-based web services. • Java and .NET examples provided • API covers • Ways to query the google database • Response formats / error responses

Google Example[7.2] public class GoogleAPIDemo { public static void main(String[] args) { … // Create a Google Search object, set our authorization key GoogleSearch s = new GoogleSearch(); s.setKey(clientKey); // do search or cache query, then print out result try { if (directive.equalsIgnoreCase("search")) { s.setQueryString(directiveArg); GoogleSearchResult r = s.doSearch(); System.out.println("Google Search Results:"); System.out.println(r.toString()); } else if (directive.equalsIgnoreCase("cached")) { …. } else if (directive.equalsIgnoreCase("spell")) { … } else { printUsageAndExit(); } } catch (GoogleSearchFault f) { System.out.println("The call to the Google Web APIs failed:"); System.out.println(f.toString()); } } private static void printUsageAndExit() { … } }}

Google Summary • Simple API focused on single function • Querying the google database. • Loose coupling • Google doesn’t care about you, or what you’re doing. • Coarse-grained • Here’s a query, and there’s your answer. No details beyond those needed for business function. • These two characteristics are core to almost all ‘Web’ development models.

D. Jabber: Messaging & presence • Presence • User directory • Proxys to Yahoo, ICQ • Other services Jabber clients Jabber server Jabber server Jabber servers

Jabber Example Jabber client [8.1] Jabberclient Jabber client Jabber server Jabber server • Requests and responses all sent in XML • Generic XML protocol for exchanging messages, plus some services. • Can be extended to non-text messaging applications • Now used for virtual presence: lets you know who else is at a blog or other web site • Connect register presence • Lookup user contact database • Send text message contact database

Web Server registers with virtual presence (VP) server, using a URL VP client gets name of VP server (URL) from Web server. VP client registers with VP server. VP server tells VP client who else is there. VP client provides chat service to other users of the site. Virtual Presence(Jabbber extension: http://www.lluna.de/ [8.2]) VP server 1 WS 3 2 4 VP client 5

Key features • Simple, single-purpose • URL-based referencing scheme • Underlying protocol uses XML • Loose coupling between components • servers, clients – failure is expected • Less-simple protocol / API • May explain why it isn’t as popular as other technologies.

And many other examples • Bittorrent - file sharing / transfers [9] • http://bitconjurer.org/BitTorrent/ • Wikis collaborative web authoring • http://en.wikipedia.org/wiki/Main_Page • http://en.wikipedia.org/wiki/Wiki [10] • …

An example application • Web page (from web server) that includes data from an aggregator, • uses google to provide searching • publishes new content info using RSS • Uses jabber to provide presence functionality to visitors • Provides bittorrent links to downloadable data • All integrated using URLs, and simple web protocols.

Conclusions • Social  focus on community, networking, connection, sharing • Technical  Software design focused on • Simplicity of interfaces, and applications • URL as the key reference for an application, function, method • Loose coupling between applications • Construction of Web applications by integrating these different ‘application services’ together.

Conclusion II “A large part of how we think about music is influenced by the methods by with which it has conventionally been distributed. We think of pop songs as being three or four minutes long because 40 years ago that was all that could fit on one side of a vinyl single.” Moby We often think of Internet-based computing is the same way -- in terms of what we know -- not what it is becoming. It is different, and challenges our thinking! Couldn’t be a more fun time to be doing software!

Blogging, Googling, Syndication: What’s the Web coming to? --- The End --- Ian Graham Enterprise IT Strategy & BMO Connect, T&S T: 416.513.5656 E: ian.graham@bmo.com W: http://www.utoronto.ca/ian/talks/

Some Tools and Sites • Blogging software • http://www.moveabletype.com • http://www.livejournal.org/ • http://manila.userland.com/ • http://www.slashcode.com/ • Free blogging sites • http://www.blogger.com  http://newrandom.blogspot.com (Ian’s site) • http://www.livejournal.com • RSS / Atom Aggregation sites • http://aggregator.weblogs.co.uk/ • http://www.bloglines.com/ • http://www.syndic8.com/ • http://meerkat.oreilly.com (Open Wire Service) • API Documentation: http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html • RSS / Atom Desktop clients • http://www.newzcrawler.com • http://www.headlineviewer.com

Some Tools and Sites • Google tools and APIs • Google Hacks, Calishain and Dornfest, O’Reilly and Associates, 2003 • http://www.oreilly.com/catalog/googlehks/index.html • http://searchenginewatch.com/searchday/article.php/2161301 • http://www.google.com/faq_freewebsearch.html (Free websearch) • http://www.google.com/services/custom.html (custom search) • http://cyberzoide.developpez.com/html/google/ • http://www.google.ca/apis/ (Google Web service APIs) • Jabber • http://www.jabber.org • http://www.lluna.de (virtual presence extension) • Bittorrent • http://bitconjurer.org/BitTorrent/ • Wikis • http://en.wikipedia.org/wiki/Wiki (all about Wikis) • Java Development Kit • http://java.sun.com/j2se/1.4.2/download.html

Blogging, Googling, Syndication: What’s the Web coming to?