1 / 20

Dynamic Distributed Services Architecture at ACAT 2003

This presentation by Iosif Legrand at the California Institute of Technology outlines a dynamic distributed services architecture with a hierarchical structure of loosely coupled and autonomous entities that dynamically cooperate using proxies or self-describing protocols. It emphasizes adaptability, scalability, and avoiding single points of failure in a large-scale network environment. Key topics include proxy services, dynamic code loading, monitoring data collection, registration/discovery mechanisms, and mobile agents for data replication. The presentation also covers service registration, lease mechanisms, and network connectivity strategies.

brae
Download Presentation

Dynamic Distributed Services Architecture at ACAT 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Iosif Legrand California Institute of Technology ACAT 2003 Iosif Legrand

  2. Distributed Dynamic Services Architecture • Hierarchical structure of loosely coupled services which are independent & autonomous entities able to cooperate using a dynamic set of proxies or self describing protocols. • They need a dynamic registration and discovery & subscription mechanism • For an effective use of distributed resources, these services should provide adaptability and self-organization (aggregation and hierarchical orchestration) • Reliable on a large scale network distributed environment • Avoid single points of failure • Automatic re-activation of components and services • Scalable & Flexible for adding dynamically new services and automatically replicate existing ones to cope with time dependent load ACAT 2003 Iosif Legrand

  3. Lookup Stub Service Skeleton Lookup Service Distributed Object Systems CORBA , RMI, DCOM “Traditional” Distributed Object Models (CORBA, DCOM) The Stub is linked to the Client. The Client must know about the service from the beginning and needs the right stub for it Server CLIENT “IDL” Compiler The Server and the client code must be created together !! ACAT 2003 Iosif Legrand

  4. Interface Lookup Service WSDL Lookup Service Web Services WSDL/SOAP SOAP Server CLIENT The client can dynamically generate the data structures and the interfaces for using remote objects based on WSDL Platform independent ACAT 2003 Iosif Legrand

  5. Lookup Proxy Proxy Lookup Service Service Any well suited protocol for the application Service CLIENT Dynamic Code Loading Services can be used dynamically • Remote Services Proxy == RMI Stub • Mobile Agents Proxy == Entire Service • “Smart Proxies” Proxy adjusts to the client MonALISA Design Considerations Act as a true dynamic service and provide the necessary functionally to be used by any other services that require such information (Jini, interface to WSDL / SOAP) • mechanism to dynamically discover all the “Service Units" • remote event notification for changes in the any system • lease mechanism for each registered unit ACAT 2003 Iosif Legrand

  6. CLIENT Lookup Lookup Service Service Lookup Lookup Service Service Register Service ID Register with ID Service Ask for a lease Get a lease for DT JINI – Network Services A Service Registers with at least one Lookup Service using the same ID. It provides information about its functionality and the URL addressed from where interested clients may get the dynamic code to use it. The Service must ask each Lookup Service for a lease and periodically renew it. If a Service fails to renew the lease, it is removed form the Lookup Service Directory. When problems are solved, it can re-register. jar jar Web Web Server Server Publish the Publish the “Interface” jar “Interface” jar The lease mechanism allows the Lookup Service to keep an up to date directory of services and correctly handle network problems. ACAT 2003 Iosif Legrand

  7. Monitoring: Data Collection Dynamic Thread Pool Other tools (Ganglia, MRT…) PULL SNMP get & walk rsh | ssh remote scripts End-To-End measurements Farm Monitor • Configuration Control Trap Listener PUSH snmp trap WEB Server Dynamic loading of modules or agents Trap Agent (ucd – snmp) perl ACAT 2003 Iosif Legrand

  8. MonALISA Service MonALISA Service MonALISA Service Registration / Discovery / Remote Notification Registration Discovery Client (other service) Lookup Service Services Proxy Multiplexer Data Filters & Agents Services Proxy Multiplexer Lookup Service Client (other service) ACAT 2003 Iosif Legrand

  9. Service Monitor UNIT & Data Handling Lookup Service Monitor Data Stores Lookup Service Client (other service) Web client WEB Service WSDL SOAP Farm Monitor Data Cache Service & DB Discovery Registration Client (other service) Java data McKoi DB MySQL Predicates & Agents Other tools • Configuration Control (SSL) UDP MySQL • Predicates & Agents User defined loadable Modules to write /sent data MDS ACAT 2003 Iosif Legrand

  10. @ CALTECH Global Client for Farms and Network Connectivity DataTAG - - ACAT 2003 Iosif Legrand

  11. Mobile Agents and Filters Simple “Global Load” filter agent From FNAL to all Maximum Flow Data Replication Path Agent Deployed to each RC and evaluates the best path for real-time data replication From CERN to all ACAT 2003 Iosif Legrand

  12. Lookup Service MonaLisa Service Discovery WAP • Filter Agents / Data TOMCAT JSP/servelts Pseudo Client WEB MySQL IDB MySQL IDB MySQL • Filter Agents / Data MonaLisa Service Lookup Service Pseudo – Clients & Dedicated Repositories ACAT 2003 Iosif Legrand

  13. VRVS Architecture pub cal- tech cor- nell funet Reflectors are hosts that interconnect users by permanent IP tunnels. vrvs 5 star- light vrvs us vrvs eu The active IP tunnels must be selected so that there is no cycle formed. usf sinica inet 2 Tree usp The selection is made according to the assumed network links performance. kek triumf ACAT 2003 Iosif Legrand

  14. Barůvka‘s Algorithm 5 A B 4 6 2 2 D C 3 1 2 3 E F 4 ACAT 2003 Iosif Legrand

  15. Barůvka‘s Algorithm 5 A B 4 6 2 2 D C 3 1 2 3 E F 4 ACAT 2003 Iosif Legrand

  16. Barůvka‘s Algorithm 5 A B 4 6 2 2 D C 3 1 2 3 E F 4 ACAT 2003 Iosif Legrand

  17. Monitoring VRVS Reflectors ACAT 2003 Iosif Legrand

  18. Global Client / Dynamic Discovery ACAT 2003 Iosif Legrand

  19. MonALISA repositories ACAT 2003 Iosif Legrand

  20. SUMMARY • MonALISA is able to dynamically discover all the “Service Units" used by a community and through the remote event notification mechanism keeps an update state for the entire system • Automatic & secure code update (services and clients) . • Dynamic configuration for services. Secure Admin interface. • Access to aggregate farm values and all the details for each node • Selected real time / historical data for any subscribed listeners • Active filter agents to process the data and provided dedicated / customized information to other services or clients. • Mobile Agents for decision support and global optimization. • Dynamic proxies and WSDL & WAP pages for services. • Embedded SNMP support and interfaces with other tools ( LSF, PBS, Ganglia, Hawkeye, IEPM-BW…) • Dedicate pseudo-clients for repository, WAP access or decision making units • It proved to be a stable and reliable distributed service system. It is currently running at ~ 100 sites • http://monalisa.cacr.caltech.edu ACAT 2003 Iosif Legrand

More Related