A Modular Data Pipelining Architecture (MDPA) for enabling Universal Accessibility in P2P Grids

1. A Modular Data Pipelining Architecture (MDPA) for enabling Universal Accessibility in P2P Grids Sangmi Lee Department of Computer Science Florida State University July. 03. 2003

2. Outline Part I. Introductory Concepts and Motivation Part II. Related Works Part III. Enabling universal accessibility in Services Part IV. Prototype of MDPA Part V. Contributions and Future Work slee@csit.fsu.edu

3. Part I. Introductory Concepts and Motivation slee@csit.fsu.edu

4. Some Observations for Distributed Services • Distributed Object Computing and Resource Utilization over the Internet. • Grids manage and share asynchronous resources in a rather centralized fashion • Peer-to-peer networks are “just like” Grids with different implementations of services like registration and look-up • Computers are fast and getting faster. One can afford many strategies that used to be unrealistic • All messages can be publish/subscribe • Software message routing • Need Synchronous and Asynchronous Resource Sharing • Can provide universal access using synchronous collaboration technology • Web Services interact with messages • Everything (including applications like PowerPoint will be a WS?) • XML will be used for most interesting data and meta-data • P2P Grid provides P2P network between resources or users in a Grid environment slee@csit.fsu.edu

5. Database Database P2P P2P P2P P2P P2P P2P P2P P2P Classic Grid P2P Grid Resources Middle TierBrokers Service Providers Composition Content Access Netsolve Security Computing Collaboration Clients slee@csit.fsu.edu

6. Universal Accessibility for P2P Grid • Emerging Miniaturized Devices • Wireless Network Communication (Wireless LAN(802.11b), 2.5, 3G wireless technology, PAN (Bluetooth), Satellite Communication, etc.) • Widespread use of Mobile Devices – Affordable mobile devices and services • Tendency of extending service for mobile devices. • As a user: Mobile devices can be users of traditional distributed services • As a service provider: A mobile device can itself be a service provider with its portability (e.g. weather or traffic sensor and send information with wireless network) • However problems pertaining to limited computing and display capabilities in addition to unreliable network communications persist. slee@csit.fsu.edu

7. Our Experiences I • We’ve investigated on the Distributed Collaborative Services: Tango Interactive (NPAC, Syracuse Univ.) Garnet Collaborative Service (FSU) • Garnet Collaborative Service • Purpose : Support distance Education, Training and if possible Computing as Grid(Web) Services • Integrate Synchronous and Asynchronous collaboration • Support universal access including PDA’s collaboration with desktops • Uniform XML event (message) based architecture • All data structures defined in an XML Schema called GXOS • XML for all metadata (Users, documents, computers) and object changes -- from text chats to display changes etc. • MyXoS manipulates GXOS objects • We build on GMS/JMS (Java Message Service) as industry standard to implement publish/subscribe model • Support collaborative features : basic interactive features (textchat, whiteboard, etc.), shared resources (shared display, shared export), AV conferences. slee@csit.fsu.edu

8. Our Experiences II : Short Demonstration slee@csit.fsu.edu

9. Our Experiences III: PDA Adaptor (Personal Server) slee@csit.fsu.edu

10. Lessons from Earlier works • Adaptor architecture provided us an easy way to integrate PDAs to existing distributed systems. • However, we faced the limitation of proxy architecture. • Content adaptation concentrated on adaptor could cause inefficient network bandwidth usage. • Proxy-based architecture for integrating network communication protocols could not ensure end-to-end service features of intelligent messaging frameworks. • Optimization of individual messages could be an overhead on message delivery • The wireless communication capability and computing power of mobile devices are getting better and better. slee@csit.fsu.edu

11. Motivation • A P2P Grid provides a natural environment for heterogeneous resource sharing • A traditional 3-tier distributed service expects unified user devices with the infrastructure focused on back-end resources or the middleware messaging service. • Early efforts • Software architectures for user interface (Seeheim model, MVC, PAC, etc) which have been developed to support various advanced approaches in user interface design. • Scientific Visualization, which maps raw data sets to a graphical form for effective human visual senses. These efforts (IRIS, AVS…) were developed for a specific purpose and are not adaptable into a general purpose distributed service as it is.  The idea of separating user presentation from raw data is still a major approach in distributed services such as the Web service infrastructure. • Therefore, the P2P Grid needs a software architecture, for supporting heterogeneous devices, which clearly defines data processing within the services that it provides. slee@csit.fsu.edu

12. Research Objectives • Investigating a software architecture which provides, • Modular data processing • Flexible Interoperability between object processing • User Interactivity • Universal Accessibility • Developing a prototype based on this software architecture • Ability to apply this software architecture design to other areas • A service paradigm, which factors both the nature and capability of heterogeneous devices into its design. slee@csit.fsu.edu

13. Part II. Related Work slee@csit.fsu.edu

14. Classical Approaches for Presentation Generation • PAC-Amodeos model : Presentation-Abstraction-Control (PAC) model + Arch model( refined Seehiem model ) • MVC (Model-View-Controller) Architecture • Advantage : Separating User Presentation from Raw data object  Provides flexible data representation (multiple synchronized views on the same information) • Open problems : Data processing is defined coarsely. As a consequence, the design of the adapting process in multiple service instances cannot be unified  cannot ensure interoperability between data processing • Approaches in Scientific Visualization : Focused on the specific purpose of application, thus, hard to adapt into general purpose applications. slee@csit.fsu.edu

15. Proxy-based architectural approaches • Provides an interface to the service, which adapts mobile devices to the conventional network communication environments. (iMobile from AT&T, WBI project from IBM, ICEBERG from Berkeley) • Advantages • More manageable for a specific group of users • Easy to integrate new devices to legacy service. • Disadvantages • Network communication overhead if the content adapting scheme is performed only in proxy. • Hard to ensure end-to-end service for existing communication framework (security, fault-tolerance, etc) slee@csit.fsu.edu

16. Standards for Heterogeneous Environments • W3C(XML based markup languages, CC/PP..) OASIS (Web service description language), TEI, UnicodeConsotium, ANSI, Dublin Core Metadata Initiate etc. • Advantage • Interoperability between heterogeneous devices, applications and network environments. • Open Problems • In some specific area, “Data Standard” does not have meaning of standard because of too many standards—limited interoperability within the group using same metadata sets. • Complexity of data processing—parsing, indexing, searching. slee@csit.fsu.edu

17. Content Adapting Technologies • Transcoding : Data process which mutates and customizes an object based on the requesting device and the user’s preferences • Web Clipping (Palm), Quality Aware Transcoding (Duke Univ.) TranSqui (Univ. of Massachusetts), Web Sphere Everyplace Access (IBM) • Transcoding technology can be a remote resource in distribute services. slee@csit.fsu.edu

18. Part III. Enabling universal accessibility in Services slee@csit.fsu.edu

19. Modular Data Pipelining Architecture (MDPA) • Basic Constraints for the software architecture for enabling Universal Accessibility • Modularity • Adaptability • User Interactivity • Flexibility • Reusability • Scalability • Interoperability slee@csit.fsu.edu

20. Modular design of Data Processing • DAT (Data) stage: Data source (facing remote resources) • CTL (Control Logic) stage: Semantic control of the data • DTX (Data Transformer) stage: Process of filtering the object for heterogeneous devices • PCL (Presentation client) stage: Generating abstract presentation for each client • MCL (Minimal client) stage: Actual drawing onto user device slee@csit.fsu.edu

21. Event Model in MDPA (I) • MDPA is an event based software architecture • Two directions of dataflow • Event transmission • Presentation Generation slee@csit.fsu.edu

22. Event Model in MDPA (II) MCL event • Different stages have different data processing • User Event is a combination of multiple event types or single event type : e.g. loading new URL includes moving mouse to text window (MCL) + Typing text (MCL) + Loading new data (EXT) • We define minimal unit of user event based on the pattern of event processing. • We should show MDPA supports these event types successfully. EXT event DAT event CTL event DTX event PCL event slee@csit.fsu.edu

23. Architecture of Each Stage • Interface: A set of ports facing other stages, users, or resources. • Presentation Filters: Processes for generating user presentations. • Event Filters : Processes for transforming user event to deliver to next stage based on the original information • Stage Manager : Basic concurrency management. • Finally, Event type is decided by which stage executes its Event Processor. slee@csit.fsu.edu

24. Modeling of MDPA • Using software architecture pattern of “Pipes and Filters” : suitable for presenting data flow • Each processing step is encapsulated in a filter component • Data is passed through pipes between adjacent filters. • Recombining these filters allows us to build families of related filters. • Representation of data process in MDPA : Modular PT-nets (Place and Transitions petri-net) • The main uses of Petri nets are for discrete event simulation and modeling complex system such as communication protocols or distributed databases. • State-based formalism • Graphical expressiveness • Well developed tools slee@csit.fsu.edu

25. Place A Transition T Place B Modular PT-net token • Original PT-net includes four components: a set of place P, a set of transitions T, a weight function W, and a mapping function for the initial marking Mo. • Modular PT-net : sharing Place or Transition, so, easy to represent Modular approach for big systems [Christensen+00]. • Each module should be able to generate output independently. • Each stage of MDPA is represented as a Module and sharing transitions (interface between stages) Place A Fusing Transition T Place B Module A Module B Two modules sharing one transition slee@csit.fsu.edu

26. A Service with MDPA • A Service : A set of processes from capturing user event to delivering user presentation • Defining of a Service within MDPA following the Definition of Modular PT-net • Define Stage as a Module of Modular PT-net • There is no fusing Place • Define Interface as a fusing Transitions  Put it altogether! slee@csit.fsu.edu

27. Definition of a Service Define a Service architecture following Definition of Modular PT-net Def. 4.7 ~ 4.11 slee@csit.fsu.edu

28. Presentation Filters Event Filters Fusing transitions : Interface Stage Manager Event Processor Representation of Stage with Modular PT-net Definition 4.7 ~ 4.11 slee@csit.fsu.edu

29. Tools for Modeling and Analysis • Demonstration : Visual Object net++ Technical University of Ilmenau, Germany http://www.systemtechnik.tu-ilmenau.de/~drath/visual_E.htm • Analysis : Tina Ver. 2.5.1 LAAS ( Laboratory for Analysis and Architecture of Systems), France http://www.laas.fr/tina/announce-2.5.1.html slee@csit.fsu.edu

30. Analysis of Modeling MDPA • Specification : Specify every path for every event types (Table 4.2)  specify every paths inside of stage for each event • Implementation : Defining a Modular PT-net (Def.4.7 ~ 4.12) • How to verify : Specification .iff. Implementation ? • We analyze our implementation with PT-net tools. • Show the specification is satisfied with our PT-net implementation (Is each path strongly connected?) • SCC (Strongly Connected Components) : Two nodes are strongly connected, iff, there exists a finite directed path, which starts at a source node and ends in a destination node. • Finally, each path specified in the stage (module) is strongly connected. • Obviously, since we used fusing transitions for representing interfaces, each interface is strongly connected. • No formal verification is included in this dissertation. slee@csit.fsu.edu

31. Aggregation of Services • Integrating services and enabling multiple service instances for user • Interface facing each data pipeline • Shared process of presentation generation • Independent event transmission to each data pipeline • Unified MCL stage for a single device Service Instance A Aggregated Service Service Instance B slee@csit.fsu.edu

32. Collaborative service with MDPA • MDPA considers the interoperability between users in P2P Grid environments. • Asynchronous collaboration • Synchronous collaboration (real-time collaboration, 10 ~ 1000 msec.) • Shared Display : Sharing image data in the master’s framebuffer • Shared Input Port : Sharing user input event and generating individual display with the full set of data processing. • Shared Output Port : Sharing user output, more flexible than shared display because it is customizable. • The collaboration model within MDPA aims to support a flexible collaborative environment for heterogeneous users. slee@csit.fsu.edu

33. Shared Display model • Immediate sharing of the bitmap image data from the master user’s framebuffer memory • Basic filtering (compression data, etc) is performed in the Messaging middleware • Master/Participants share all stages except for the MCL stage • Therefore easy to implement. However, this is less flexible. slee@csit.fsu.edu

34. Shared Output Port Model Sharing Output port to PCL stage Basic content adaptation is possible Each participant should have PCL and MCL stages slee@csit.fsu.edu

35. Shared Input Port Model (CTL Event) Every user has full set of MDPA Only sharing user input event For each shared input event each service generates user presentation slee@csit.fsu.edu

36. Shared Input Port Model (DAT/EXT Event) DAT event EXT event slee@csit.fsu.edu

37. Application –Aware Transcodingfor Shared Display Desktop PC LAN/J2SE 1106 x 930 4017.9 KB Bitmap Image • Goal : Fast and Efficient Propagation upon change of the master’s display. • Approach: Reduces size of data over the wireless network • Graphic Format Transformation for specific devices (Based on Java MDPA) • Data Compression • Color Depth Adjustment • Image Resolution Adjustment • Aggregative Update Filtering shared display message for wireless devices PDAs(iPaq3900) Resolution:240 x 320 Wireless LAN 802.11b J2ME(Personal Java) Image size:553 x 465 50.9 KB Bitmap Image SmartPhone(treo300) Resolution:160x160 3G wireless comm. J2ME(MIDP) Image size:120x100 1.7 KB/PNG image slee@csit.fsu.edu

38. Application –Aware Transcodingfor Shared Export • Goal : Support Flexible view points. • Approach: Manipulate abstracted data and Reduce size of data • Graphic Format Transformation for specific devices (Based on Java MDPA) • Stylesheets (CSS) • Data Compression • Image Resolution Adjustment Styling with CSS for Black/White PDAs Workflow of Generating image for Shared Export (Collaborative SVG browser) SVG Document Rendering Image Formatting Data Compression Stylesheet User Profile slee@csit.fsu.edu

39. Part IV. Prototype of MDPA slee@csit.fsu.edu

40. Challenge of the Web Service Architecture • Provide clear framework with the representation of resources and well defined interfaces (ports) specified in WSDL • Enable the development of collaborative services that can interoperate with existing services implemented in different ways (Perl, Phython, Java, etc.) • Support for unified user setup of collaborative environment for multiple services. • Easy to reuse/discover for integrated collaborative features. • Support for object sharing and integrated service environments slee@csit.fsu.edu

41. WSDL WSDL WSDL WSDL W W W W S S S S Collaborative Application Service/stage Collaborative Application Collaborative Application R R R R P P P P Web Service Web Service Web Service Web Service Developing MDPA Service as a Web Service • Each service is encapsulated as WS semantics • Interfaces between services, users and resources are defined clearly in WSDL • Each WS is accessible to other WS via a resource facing port. • Multiple services are integrated into a Portlet aggregation environment (Jetspeed, WebSpheres) • Accessible to other resources (including DB, file systems, URL, 3rd party services, A/V conference services, etc.) via resource faced “ports”. • Flexible packaging method is possible Collaborative Applications as Web Services: defined with WSDL User Face Web Service WSRP Ports : define WS as a Portlet slee@csit.fsu.edu

42. Remote resources Example 1. Put it all together in one WS WSDL W W W W W W S S S S S S DAT CTL DTX R R R R R R P P P P P P Web Service Remote resources Example 2. One Stage in One WS WSDL WSDL WSDL WSDL WSDL DAT CTL DTX WS WS WS WS WS Remote resources DTX DAT WS WS WS CTL Example 3. Accessing multiple WSs in one stage Packaging and Structuring the service slee@csit.fsu.edu

43. Carousel Web Service • Prototype of Real-time collaborative service with MDPA. • DAT+CTL+DTX is designed as a Web Service (example.1) • PTL is implemented in two phases • Markup Languages (HTML/WML) • Layout of Client application (J2SE AWT APIs/J2ME MIDP) • MCL is supported in various ways (Traditional PC monitor(32bit true color)/ iPaq(320X240 12bit color)/ Treo300(160x160 8bit color)) based on J2ME technology • Real-time collaboration requires efficient and reliable messaging framework slee@csit.fsu.edu

44. Distributed Event Service :NaradaBrokering • Based on a network of cooperating broker nodes • Cluster based architecture allows system to scale to arbitrary size • Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. • Now has four major core functions • Message transport (based on performance measurement) in heterogeneous multi-link fashion • General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing • Filtering for heterogeneous clients • Federation of multiple instances of Grid services slee@csit.fsu.edu

45. Collaborative WebService Collaborative WebService Collaborative WebService Collaborative Web Services Linkage with Event Service Remote Collaborative Web Services : integrated As portlets in Portal Aggregator Real-time event is processed By the event service (NaradaBrokering) : text message/framebuffer image/URLs/Microsoft events, etc. Portal Aggregator Event Service Heterogeneous Users slee@csit.fsu.edu

46. Session Manager Architecture of CAROUSELWeb service Collaborative Web Services Collaborative Web Services Collaborative Web Services NB publish/subscribe Portlets Event Service Portal Aggregator HTTP HTTP NB publish/subscribe Traditional Desktop devices Handheld wireless devices slee@csit.fsu.edu

47. Scenario Aggregative Portal Session Manager 1. Login 2. Setup 4. User preference/ service request 3. User setup input 5. Service Topics / Available NB Information 7. Service Topics NB information 6. Service Invocation user 8. User Event 9. Service Output Collaborative WS NaradaBrokering slee@csit.fsu.edu

48. Security Framework • Required in both phases of network communication • Within Traditional WS Architecture • (Web Service Portal, Web Service  Web Service) • Popular Security Solution is possible (SSL, SOAP Security Extentions (W3C), S-HTTP, etc) • Within Real-time messaging • Inherit Security Framework of NaradaBorkering • Topic based encryption of message exchanges • Key Management Center (KMC) manages key distribution • Ensures end-to-end message security • Mobile device should have encryption/decryption module if it is the generator/consumer of secure messages respectively. slee@csit.fsu.edu

49. Reliability and Fault Tolerance • Reliability and Fault Tolerance also should be considered in Two Phase of network communications • Within the traditional WS Architecture • Portal service for Mobile Device is stateless. • Within the Real-time Messaging Service • Inherit reliability and fault tolerance scheme from high-level messaging such as JMS scheme (NaradaBrokering is JMS compliant) • Access point for the mobile device should provide reliability. Application layer protocol can ensure the reliability (e.g. message acknowledgements, status of network communication links, etc) slee@csit.fsu.edu

50. Shared Input Port Model with Web Service Infrastructure WSViewer WSDisplay R R R U U U F F F F F F WebService WebService WebService I I I I I I O O O O O O WS Viewer WS Display WSDisplay WS Viewer Collaboration as a WSSet up Session Master Event Service OtherParticipants slee@csit.fsu.edu