Collaboration and Web Services

1. UIUC October 15 2002 Collaboration and Web Services PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 (Technology Officer, Anabas Corporation, San Jose) http://grids.ucs.indiana.edu/ptliupages gcf@indiana.edu uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

2. Some Basic Observations/Goals • Technology Support for e-learning is one motivation • Need Synchronous and Asynchronous Resource Sharing • Can provide universal access using synchronous collaboration technology • Grids manage and share asynchronous resources in a rather centralized fashion • Peer-to-peer networks are “just like” Grids with different implementations of services like registration and look-up • Web Services interact with messages • Everything (including applications like PowerPoint will be a WS?) • Computers are fast and getting faster. One can afford many strategies that used to be unrealistic • All messages can be publish/subscribe • Software message routing • XML will be used for most interesting data and meta-data • One will store/consider data and meta-data separately but often use same technology to manage both of them. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

3. Deductions • The system consists of a sea of message-based Services • Services inject and extract messages whose transport and manipulation is support by a logically distinct sea of brokers/routers • They support adaptive routing, filtering, workflow … • They separate logical and actual transport • These form a federated XML database and support asynchronous collaboration • These process real-time messages in about a millisecond and support synchronous collaboration • Basic Unit of information (including events and messages) is a bunch of XML using URI’s to link to other XML or to “other technology capabilities” – Fortran programs, video files, telescopes … • This XML includes service meta-data, user profiles, H323 done right, your homework grades, update of framebuffer in shared display etc. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

4. So what are we doing I? • We need to be UIUC to do all of this and I can only do a few scattered projects based on this model • We have designed and built a messaging infrastructure NaradaBrokering embodying some of these ideas • We have shown interoperability between JXTA (Sun’s P2P environment), Java Message Service (JMS) and NaradaBrokering • We have deployed a classic (Placeware, Interwise, WebEx) synchronous collaboration environment (Garnet) using JMS or Narada (uses Anabas technology) • We have illustrated filtering/universal access by linking PDA’s to desktop collaboration • We have prototyped audio-video conferencing as a web service • We are repackaging collaborative SVG as a Web service to illustrate (explore) how wonderful it will be when all applications are Collaborative Web services uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

5. So what are we doing II?but won’t mention again • We are building a toolkit to allow any application to be made a Web service (this only manipulates metadata) • Aimed at support of classic Grid applications running on a backend supercomputer • A Computing Portal • We are building Jetspeed portlets of various types – including collaborative • We have some XML news group technology which uses news group interface to support management and browsing of multiple XML information nuggets (instances) of general Schema • Wizard generates interfaces to define nuggets • JSP wizard layout and browsing specification defined by annotating Schema uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

6. Database Database Classic Grid Architecture Resources Content Access Composition Middle TierBrokers Service Providers Netsolve Security Collaboration Computing Middle Tier becomes Web Services Clients Users and Devices uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

7. Different Web Service Organizations • Everything is a resource (distributed object)implemented as a Web Service, whether it be: • back end supercomputers and a petabyte dataset • Microsoft PowerPoint and this file • Web Services communicate by messages ….. • Web Services are “just distributed objects” with focus on (particular) XML specified input and output messages • Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport ….. • Gives a Peer-to-Peer Grid • Roughly but not completely consistent with OGSA • Consistent with “rule”: build everything as a Web service uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

8. Database Database Event/MessageBrokers Event/MessageBrokers Event/MessageBrokers Peer to Peer Grid Peers Service FacingWeb Service Interfaces Peers User FacingWeb Service Interfaces A democratic organization Peer to Peer Grid uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

9. Education as a Web Service • “Learning Object” XML standards already exist from IMS/ADL http://www.adlnet.org – need to update architecture • Web Services for virtual university include: • Registration • Performance (grading) • Authoring of Curriculum • Online laboratories for real and virtual instruments • Homework submission • Quizzesof various types (multiple choice, random parameters) • Assessment data access and analysis • Synchronous Delivery of Curricula including Audio/Video Conferencing and other synchronous collaborative tools as Web Services • Scheduling of courses and mentoring sessions • Asynchronous access, data-mining and knowledge discovery • Learning Plan agents to guide students and teachers uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

10. Collaboration and Web Services • Collaboration has • Mechanism to set up members (people, devices) of a “collaborative sessions” • Shared generic tools such as text chat, white boards, audio-video conferencing • Shared applications such as Web Pages, PowerPoint, Visualization • b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment • We can port objects to Web Services and build a general approach for making Web services collaborative • a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we can make it a WS quite easily uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

11. Web Service Architecturefor Audio Video Conferencing uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

12. XGSP: Introduction Collaboration as a WS • Registration Method registration server with its alias name and current location • Session Command Method Membership Control Commands, Session Control Commands • Query Method discover various properties about the system • Session Channel Binding Method (Specific to A/V) bind the RTP channels of a client into the media server uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

13. XGSP: Example <SessionDes> <SessionName> PervasiveTech Seminar </SessionName> <SessionID> 1234567 </SessionID> <SessionCreator> Ahmet@indiana.edu </SessionCreator> <SessionInfo> this is a meeting on the XGSP </SessionInfo> <SessionPlace> Lobby Room </SessionPlace> <SessionTime> <StartTime> (EastTime) 10:00AM </StartTime> <EndTime> (EastTime) 12:00AM </EndTime> </SessionTime> <SessionURI> http://grids.ucs.indiana.edu/~ag </SessionURI> <SessionParticipants> <Participant> Wenjun@156.56.103.129 </Participant> <Participant> Hasan@156.56.103.27 </Participant> <Participant> Shrideeper@156.56.103.111 </Participant> </SessionParticipants> <ContactInfo> wewu@indiana.edu </ContactInfo> </SessionDes> uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

14. Linking Clientsand Servers Current ImplementationPolycom (H323) Access Grid Integration Future ProjectLink Proprietary MCU’s Illustrated for SIP (HearMe)and Access Grid uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

15. Current Status • XGSP Specification stable • Demo prototype of Polycom (H323), Access Grid, Shared Display clients • H323 Gateway based on openh323 JMF (Java Media Framework) used for Media Server XGSP used Internally between audio, video and session control services Codec negotiation supported (No XGSP clients yet) Narada UDP Communication has been successfully tested XGSP MCU (Control) User Interface uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

16. H323 Client (Polycom) in XGSP Session uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

17. Comparison with other approaches uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

18. Possible A/V Web Service Futures • Productize Narada Integration – needs more testing of Narada multi-protocol interface • Will this defeat firewalls that currently spoil my lectures? • Session Control Server can be used to define collaborative sessions for other shared applications • Text chat etc. • Shared SVG, StarOffice, Internet Explorer, Word etc. (using “.net” event interface) • Integrate with JXTA interface using Narada-JXTA link • Add RealMedia (Windows Media) SIP (VOIP) and native XGSP clients • Integrate at server (MCU) level for Polycom, AccessGrid, VOIP – this is Federation uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

19. Personal View of Collaborative Systems • Originally (at Syracuse) built a collaborative environment Tango and used for online lectures and training sessions from 1997-2001 (2001 updated system) • Functionality fine but browser interface and network produced instability • Hard to build custom collaborative applications • Newer simpler (Garnet) system addresses these issues although network issues still being addressed • Use of XML meta-data and building Education or e-learning as a set of Web Services gives interoperability • Use of Grid (as from IBM)and peer-to-peer technology will give improved infrastructure (still quiteprimitive) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

20. Courses at Jackson State • Taught using Tango/Garnet since fall 97 over Internet twice a week from Syracuse/FSU/Indiana • Course material same style as online material for in-university classes • Curricula, Homework, Grading, Facilities done by “me” • Students get JSU NOT Syracuse/Indiana Credit • “Trivial” in that learning model identical to that in traditional courses – just changed interaction mode • Jackson State major MSI (Minority Serving) University with many computer science graduates • Do not compete with base courses but offer addon courses with “leading edge” material (Web Technology, modern scientific computing) which give JSU (under)graduates skills that are important in their career • Fall 99 Semester CPS640 offered to 40 students in 5 distant places and separately 40 at Syracuse uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

21. Lessons from Past • “True” Shared Event flexible but too much work for most cases • Shared Display – nearly always works • Shared Export – SVG PDF HTML X3D etc. can be re-used • Integrate synchronous and asynchronous collaboration • Do not build into browser as not a reliable or standard virtual machine • Build around uniform publish-subscribe style XML Event Model supporting archiving, customization, filtering – apply to all collaboration modes • Later we realized need to integrate with Peer to Peer and Web Services (straightforward as XML event model fits these architectures) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

22. Other WS Resource orService-facingPorts User-facingPorts PortalAggregateWS-User Facing Fragments WSDL R U F F Content Provider I I O O Web Service Other WSUser FacingPorts Render User Facing Ports for Web Service uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

23. Object Viewer Object Display Object Object’ Object’’ Collaboration: Shared Display • Sharing can be done at any point on “object” or Web Service pipeline Shared Web Service SharedDisplay Shared Export Shared Event Master Shared Display shares framebuffer with eventscorresponding to changedpixels in master client. Event(Message)Service Object Display As long as pipeline uses messages, easy tomake collaborativeWindows framebuffers and in fact most applications do NOT expose a message based update interface Object Display uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

24. R R R U U U WSViewer WSDisplay F F F F F F I I I I I I WebService WebService WebService O O O O O O WS Viewer WS Display WS Viewer WSDisplay Shared Input Port (Replicated WS) Collaboration Collaboration as a WSSet up Session with XGSP Master Event(Message)Service OtherParticipants uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

25. Shared Output Port Collaboration WSDL R U Application orContent source F F WSViewer WSDisplay I I O O Web Service Collaboration as a WSSet up Session with XGSP Web Service Message Interceptor Master WS Viewer WS Display Text Chat Whiteboard Multiple masters Event(Message)Service OtherParticipants WS Viewer WSDisplay uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

26. Role of Event/Message Brokers • We will use events and messages interchangeably • An event is a time stamped message • Our systems are built from clients, servers and “event brokers” • These are logical functions – a given computer can have one or more of these functions • In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers • Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services • There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications • NaradaBrokering is designed to provide these functionalities • MPI provided such functionalities for all parallel computing uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

27. Destination Source Matching Routing Filter workflow Web Service 1 Web Service 2 (Virtual)Queue WSDLPorts WSDLPorts Broker NaradaBrokering implements an Event Service • Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor • Workflow implements message process • Routing illustrated by JXTA and includes firewall • Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism • These use Security model (being designed) based on WS-Sec uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

28. Engineering Issues Addressedby Event / Messaging Service • Application level Quality of Service – give audio highest priority • Tunnel through firewalls • Filter messages to slow (collaborative or real time) clients • Hardware multicast is erratically implemented (Event service can dynamically use software multicast) • Scaling of software multicast • Elegant implementation of Collaboration in a Groove Networks (done better) style • Integrate synchronous and asynchronous collaboration uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

29. Features of Event Service I • MPI nowadays aims at a microsecond latency • The Event Web Service aims at a millisecond (computer) latency • Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites) • Different performance/functionality trade-off • Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S • Actually a network of brokers • Synchronous systems: B acts as a real-time router/filterer • Messages can be archived and software multicast • Asynchronous systems: B acts as an XML database and workflow engine • Subscription is in each case, roughly equivalent to a (XML) database query uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

30. Features of Event Service II • In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism • All Web Services are specified in XML but can be implemented quite differently • Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP • Separate logical User Channel and its protocol from “actual ports/protocols” which are Transport Channels • Use Performance Service to map user requirements into transport • e.g. audio user channel requests UDP from A to B but we use TCP/IP on port 80 from A to N1 (through nasty firewall) and UDP from N1 to B (say on Internet2) • There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

31. Features of Event Service III • The event service is naturally implemented as a dynamic distributed network • Required for fault tolerance and performance • Implies a corresponding dynamic distributed XML database • A new classroom joins my online lecture • A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students • Company X sets up a firewall • The event service sets up brokers either side of firewall to optimize transport through the firewall • Note all message based applications use same message service • Web services imply ALL applications are (possibly virtual) message based uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

32. Data base Single Server P2P Illusion Traditional Collaboration Architecturee.g. commercial WebEx Collaboration Server uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

33. Data base Narada Broker Network (P2P) Community For message/events service Broker Broker (P2P) Community Resource Broker Broker Broker (P2P) Community Software multicast Broker (P2P) Community uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

34. Low Rate; Small Messages NaradaBrokering and JMS (Java Message Service) (commercial JMS) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

35. Narada JXTA Event Request/Response Present if targeted atParticular peer NaradaBrokering and JXTA Narada-JXTA provides JXTA guaranteed long distance delivery uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

36. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

37. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

38. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

39. NaradaBrokering Communication • Applications interface to NaradaBrokering through UserChannels. • UserChannels are implemented as TransportChannels which are made up of Links • UserChannels have publish/subscribe semantics • Links implement a single conventional “data” protocol supported by a SOAP administrative channel • Easily adding new transport protocols within the Framework. • Negotiating the best available communication protocol • Link implementations can incorporate their own handshaking protocols for setting up of the communication channel. • Different links can have different underlying transport implementations • Implementations in the next release will include support for TCP,UDP, Multicast, SSL, HTTP, HTTPS. • Support communication through proxies such as iPlanet, Netscape • Support for communication through firewalls such as Microsoft ISA. • NaradaBrokering brokers and Links can be instantiated dynamically • Support communication between two application end points across firewall & proxy boundaries. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

40. User “XMLDatabase” uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

41. Performance measurements are used by Links in Reconfiguring Connectivity between nodes Deciding underlying transport protocol Determining possible filtering TransportChannel uses “performance heuristic” to define links and their protocols Factors measured include Transit delays, bandwidth, Jitter, Receiving rates. Performance measurements are Spaced out at increasing intervals for healthy channels. Factors selectively measured for unhealthy channels. No repeated measurements of bandwidth for example. Injected into Narada network as XML events Narada Performance Web Service Administrative Interface uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

42. Firewall Control Channel (TCP) Specifics tunnel destination, parameters. [ SOAP port 80 ] Config Specifics default tunnel destination, parameters. Non-Firewall Proxy CTL SSL Tunnel Server Proxy UDP TCP SSL Tunnel Client Proxy SSL Lib API TCP UDP Firewall Proxy Fake SSL Impl. JSSE Impl. WinINET Impl. Proxy Detection API Required for MS Authentication support. Text Config WinINET Detection Required for Proxy location detection Narada Link Firewall Architecture uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

43. UDP Works Start Stream Media Types Connection Complete Doesn’t Work Start TCP Works Reliable Data Stream Doesn’t Work Windows ? WinINET Try SSL first then HTTP Works NaradaBrokeringLink Transport FirewallHeuristic Doesn’t Work Try SSL Over HTTPS Proxy Does HTTPS Proxy Exist Works Yes Doesn’t Work Try HTTP Over HTTP Proxy Does HTTP Proxy Exist Works Yes Doesn’t Work “Fake” SSL Over Direct Try SSL Over Direct Try HTTP Direct Works Doesn’t Work Doesn’t Work Works uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

44. Roles of NaradaBrokering • Originally to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous • Until recently used JMS “instead” but have replaced by NB as we can control features (bugs) in implemenation • Now has three major core functions • Message transport (based on performance) in multi-link fashion • General publish-subscribe including JMS&JXTA • Federation of multiple instances (just starting) of Grid services uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

45. NBHub Federation of Services • If you have a service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing …. • With a standard which client and server components • Then can build a “server” that services all clients • Alternatively can hierarchically consider collection of existing Server-client domains • IBM or Globus OGSA islands • Sun Grid Engine Schedulers • Polycom/Access Grid A/V sessions • Enterprise Search Engines • Federation links islands together • JXTA Search has XML specified federation approach – will try to include and generalize as a NaradaBrokering federation framework • DoD High Level Architecture HLA does this for simulation uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

46. Narada/JMS and Collaboration • Collaboration involves sharing resources and synchronous collaboration involves coordinating a common view of a resource between multiple clients • Typically one client is “in charge” and others get initial and updated resource from this “master” • Specification of initial state of resource and its change are “just XML events” and we (Anabas and Indiana) have used first JMS and now NaradaBrokering to implement the transport of update events between collaborating clients • Update events include: • text you type into text chat or Instant Messenger • URL defining shared browser • Change in framebuffer for (most flexible) shared display • Microsoft events for shared PowerPoint (file replicated between clients) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

47. Commercial CollaborationSystems Centra Anabas WebEx PlaceWare uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

48. Filtering Service • As (real-time) collaboration uses publish-subscribe, we can and in fact must map/filter “object renderings” or updates to them differently for each class of client • In distance education, needed for example to • Cope with mix of Internet2 and dial-up clients (the student who overslept and is at home) • Allow PDA’s (Personal Digital Assistants) and desktops to be in same session • To allow Blind users to select the purely audio version of some content • Mapping in event service enables universal access • Note clients’ subscription to publish/subscribe service includes “user profile” uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

49. Collaborative PDA • Extendable to more general universal access • Can implement filter either as an insertion in message stream or in batch mode where a Service subscribes to event stream (one collaborative application or “sharedlet”), filters it and reposts to a different stream • We developed first case with a special adaptor that is essentially a NaradaBrokering node that • Has added filters controlled by client profile • Has stripped down special purpose link protocol HHMS (Hand held message service) optimized for PDA • Currently implemented as MyProfessor for Windows CE iPAQ • Working on Palm OS Cell-PDA combination • Have implemented shared display, SVG, Text chat, Instant Messenger (using Jabber) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

50. Shared Display Pass Through PDA Collaboration Event Filter GMS =JMS orNarada LightweightNaradaHand HeldLink Protocol uri="http://www.naradabrokering.org" email="gcf@indiana.edu"