1 / 43

Neal Sample Stanford University

CHAIMS: A Reuse and Composition System for Services. Neal Sample Stanford University Objective: Investigate new approaches to large-scale software composition .

vance
Download Presentation

Neal Sample Stanford University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHAIMS: A Reuse and Composition System for Services Neal Sample Stanford University Objective:Investigate new approaches tolarge-scale software composition. Approach:Develop and validate a composition-only language (CLAM), a protocol for large, distributed, heterogeneous and autonomous megamodules (CPAM), and a supporting system. CHAIMS

  2. Shift in Programming Tasks Integration, Composition, Reuse Coding 1970 1990 2010 CHAIMS

  3. Composition of Processes... • versus composition and integration of Components • reusing small components (copy/paste, shared library, local install, etc.) • large distributed components within same “domain” (e.g. within one bank or airline) CHAIMS

  4. Purchasing Local Components Distributor e Domain expert, Programmers local computer or local distributed system and copy purchase I/O I/O c e a b d CHAIMS

  5. Composition of Processes... • versus composition and integration of Components • reusing small components (copy/paste, shared library, local install, etc.) • large distributed components within same “domain” (e.g. within one bank or airline) • versus composition and integration of Data • data-warehouses • wrapping data available on the web CHAIMS

  6. Data Resources Accessing Remote Data Domain expert, Programmers local computer or local distributed system I/O I/O Control & Computation Services c c l i e n t e a b d Wrappers s e r v e r s CHAIMS

  7. Composition of Processes... • versus composition and integration of Components • reusing small components (copy/paste, shared library, local install, etc.) • large distributed components within same “domain” (e.g. within one bank or airline) • versus composition and integration of Data • data-warehouses • wrapping data available on the web CHAIMS/CLAM: • compose remote autonomous services CHAIMS

  8. Domain expert Client workstation IO module IO module e b a d c Accessing Remote Services c l i e n t s data control s e r v e r s at provider's sites CHAIMS

  9. Domain expert Client workstation Domain expert Client workstation IO module IO module IO module IO module e b a d c Accessing Remote Services c l i e n t s data control s e r v e r s at provider's sites CHAIMS

  10. Domain expert Client workstation Domain expert Client workstation Domain expert Client workstation IO module IO module IO module IO module IO module IO module e b a d c Accessing Remote Services c l i e n t s data control s e r v e r s at provider's sites CHAIMS

  11. Data Resources Composing Computational Services Domain expert Client workstation c l i e n t IO module IO module CPAM and distribution systems data control s e r v e r s at provider's sites e b Remote Computation Services Megamodules a d c CHAIMS

  12. Example of Remote Services Bio-informatics: • several research institutes develop algorithms for modeling and matching molecule structures • several knowledge bases available, many of them with additional computational services • a scientist has to combine several of these processing units to produce desired result CHAIMS

  13. Typical Scenario: Logistics A general has to ship troops and/or various material from Los Angeles to Washington DC: • different kind of material: criteria for preferred transport differ • not every airport equally suited • congestion, prices • actual weather • certain due or ready dates Today: calling different companies, looking up information on the web, reservations by hand • hand-coded systems Tomorrow: system proposes possibilities that take into account various conditions • composition of processes CHAIMS

  14. Advantages Knowledge • separation of concerns (provider - composer) • technical expertise (databases …) • maintenance, newest algorithms Availability • pay per usage instead of pay per instance • lower barrier for clients to try out new modules • choice of several providers even at run-time Resources • access to computation intensive modules for everybody • minimal local resources possible Yet there are alsochallenges... Advantages of reuse by accessing remote services: CHAIMS

  15. Challenge: Computation Intensive Services What we would like: ==>exploitingparallelism among services without complex coding on client side ==>partialextraction of results when they are ready ==>progressiveextraction of results ==>monitoring progress of service invocation and results ==>cost estimation prior to invocation of a service for possibility to choose cheapest or fastest service Services are not free for a client: • execution time of a service • transfer time and fee for data • fees for services CHAIMS

  16. Challenge: Autonomy Separation of control==> client has no direct control over: • availability of services • resources provided for services Heterogeneity: • heterogeneityconcerning languages, platforms as well as distribution protocols • legacy code, applications and libraries CHAIMS

  17. Challenge: Composition Task Services must somehow be composed on the client-side • automated: generation of effective access code • composition-oriented: composition only language or interface • easy: simple for domain experts to use CHAIMS

  18. Megamodule Provider provides megamodules client side server side Wrapper Templates adds information to CHAIMS Repository b d e a c MEGA Modules CPAM protocol Distribution Systems (CORBA, RMI…) CHAIMS Architecture Composer interacts with other language or interpreter or incremental programming CHAIMS

  19. Implementation of CPAM On top of various protocols: • RMI, CORBA, DCE, plain TCP/IP, http • each primitive is a procedure call, or a pair of XML messages • or mimicked by other interfaces: SWAP, several methods Method parameters: • simple data element, or recursive structure of data elements • each data element has also type and descriptive name information • encoding: now ASN.1, future XML • either megamodule conformity or ontology mediator megamodules Wrappers for CPAM compliance: • asynchrony and state management (client-specific parameters, results) • progress information and cost estimation • data encoding CHAIMS

  20. Megamodule Provider Composer provides megamodules client side server side writes Wrapper Templates CLAM program adds information to CHAIMS Compiler CHAIMS Repository b d generates e a Client Side Run Time c MEGA Modules CPAM protocol Distribution Systems (CORBA, RMI…) CHAIMS Architecture CHAIMS

  21. Architecture: CLAM andCHAIMS-Protocols Megaprogrammer CLAM Megaprogram Compiler/Interpreter CHAIMS-protocols CORBA-idl DCE-idl Java-class M e g a m o d u l e s CHAIMS

  22. CLAM Primitives Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETPARAM: in a megamodule ESTIMATE: for optimization Invocation and result gathering: INVOKE EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a megamodule Control: WHILE, IF CHAIMS

  23. Pre-invocation: SETUP • Establish communication between client CLAM program and a megamodule • Returns a megamodule handle • Maps logical name of service to provider • ex: mm_handle = SETUP (“TransportationModule”) • CLAM programmers avoid module location and transportation protocol issues • Authentication occurs here (open question). CHAIMS

  24. Pre-invocation: ESTIMATE • Returns cost of a method invocation in terms of fee, time, and data volume • Considers values of method parameters • ESTIMATE for runtime module selection • Useful for scheduling and cost minimization. CHAIMS

  25. SETUP () SETPARAM (attributes essential for cost estimation) (f1=fee, t1=time) = ESTIMATE (“Optimum”) SETUP () SETPARAM (attributes essential for cost estimation) (f2=fee, t2=time) = ESTIMATE (“BestRoute”) time BestPick.INVOKE (“BestRoute”, …) Cost Estimation for Selection Choosing megamodules: C H A I M S c l i e n t RouteChoose - Optimum - … BestPick - BestRoute - ... calculate cost function ==> today BestPick is better CHAIMS

  26. Pre-invocation: SET-, GETPARAM • GET-: returns current parameter values • SET -: overrides current parameter values • Operates on a megamodule handle • ex: mm_handle.SETPARAM (city = “Amsterdam”) • Utility • presetting for ESTIMATE • avoid resetting with repeated invocations • simple input perturbation. CHAIMS

  27. Hierarchical Setting of Parameters Example: parameters of method TripPlan end-date start- date number from dest general default values 1JAN1998 1JAN1998 LAS BWI 1 client-specific settings for client A 4OCT1998 6OCT1998 SJO ZRH client-specific settings for client B FRA 2 method TripPlan, client A: invocation-specific settings actual values used in invocation TripPlan1 4OCT1998 6OCT1998 SJO ZRH 1 method TripPlan, client A: invocation-specific settings 7OCT1998 SFO actual values used in invocation TripPlan2 4OCT1998 7OCT1998 SFO ZRH 1 method TripPlan, client B: invocation-specific settings 1DEC1998 9DEC1998 actual values used in invocation TripPlan1 1DEC1998 9DEC1998 LAS FRA 2 CHAIMS

  28. CLAM Primitives Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETPARAM: in a megamodule ESTIMATE: for optimization Invocation and result gathering: INVOKE EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a megamodule Control: WHILE, IF CHAIMS

  29. Invocation: INVOKE • Starts execution of a method • Operates on a megamodule handle • Returns an invocation handle • ex: in_handle = mm_handle.INVOKE (“GenericMethod”) in_handle = mm_handle.INVOKE (“RoutePlanMethod”, Pair_of_Cities = cities) • Achieves asynchrony in sequential clients. CHAIMS

  30. Invocation: EXAMINE • Determines the state of a invocation • Operates on an invocation handle • Returns: • (status, “progress information”) • status = {DONE, NOT_DONE, PARTIAL, ERROR} • Utility • Progressive extractions • Partial extractions • Optimized module execution scheduling • Steering. CHAIMS

  31. Invocation: EXAMINE quick_route: found at time 30 optimal_route: found at time 100 • Method Status: WHILE (in_handle.EXAMINE() != DONE) {} (routeVar = quick_route) = in_handle.EXTRACT() • Attribute Status: WHILE (in_handle.EXAMINE(quick_route) != DONE) {} (routeVar = quick_route) = in_handle.EXTRACT() • Steering: IF (in_handle.EXAMINE(quick_route) != PARTIAL) … ask user (or scheduler) to continue or halt… CHAIMS

  32. Invocation: EXTRACT • Collects data from a megamodule method • Operates on an invocation handle • Returns: assignment of values to parameters • ex: (routeVar = quick_route) = in_handle.EXTRACT() • Allows partial data retrieval • saves volume: abandon uninteresting elements • saves time: termination of useless invocation • Allows progressive data retrieval. CHAIMS

  33. CLAM Data Types • Simple data types still have a role • opaque //exclusively used for user data • integer • string • Boolean • real • No classes or amalgamated types • Only be assigned and compared • Used for control (not opaque) CHAIMS

  34. CLAM Primitives Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETPARAM: in a megamodule ESTIMATE: for optimization Invocation and result gathering: INVOKE EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a megamodule Control: WHILE, IF CHAIMS

  35. TERMINATE • Ends method invocation or closes a connection to a megamodule • Why necessary? • delayed extractions • repeated/progressive extractions • partial extractions • early termination • ex: in_handle.TERMINATE () mm_handle.TERMINATE () CHAIMS

  36. CLAM Composition Language Purely compositional: • no primitives for arithmetic • no primitives for input/output Splitting up CALL-statement: • parallelism by asynchrony in sequential program • novel possibilities for optimizations • reduction of complexity of invoke statements Higher-level language: • assembler => HLLs, • HLLs => composition/megamodule paradigm CHAIMS

  37. Ongoing & Upcoming Research Issues c o m p o s i t i o n composition wizard optimization of compositions automatedcomposition graphicalcomposition automatedscheduling ofinvocations incrementalcomposition repositorybrowser CLAM cost estimation directoryservices Repository CPAM incrementalresult extraction electronic marketplaceof services repositoryin XML, ... exploitingXML web-services sign-up andauthentication direct dataflowsbetween megamodules CHAIMS broker e - c o m m e r c e p r o t o c o l CHAIMS

  38. Conclusion • New reuse model • computational modules stay at provider’s site • reuse of computational modules by accessing computational services over the internet • New issues • cost-estimation, progress monitoring • easy and efficient composition of remote, computation-intensive services CHAIMS

  39. Questions? http://www-db.stanford.edu/CHAIMS/ CHAIMS

  40. Other Approaches Traditional Languages/Extensions • FX: Self-contained language for module building composition. Much finer grain than service composition. • Orca, Opus: Extensions to HPF. Programmers require intimate knowledge of all process data. Composition Languages • MANIFOLD: Similar IWIM model, with black box process • Myriad scripting languages, generally assume complete ontological control of services CHAIMS

  41. Megaprogram Example: Code io_mmh = SETUP ("InputOutput") route_mmh = SETUP ("RouteInfo") ... best2_mmh.SETATTRIBUTES (criterion = "cost") cities_default = route_mmh.GETPARAM(Pair_of_Cities) input_cities_ih = io_mmh.INVOKE ("input”, cities_default) WHILE (input_cities_ih.EXAMINE() != DONE) {} cities = input_cities_ih.EXTRACT() ... route_ih = route_mmh.INVOKE ("AllRoutes", Pair_of_Cities = cities) WHILE (route_ih.EXAMINE() != DONE) {} routes = route_ih.EXTRACT() … IF (best1_mmh.ESTIMATE("Best_Route") < best2_mmh.ESTIMATE("Optimum") ) THEN {best_ih = best1_mmh.INVOKE ("Best_Route", Goods = info_goods, Pair_of_Cities = cities, List_of_Routes = routes, Cost_Ground = cost_list_ground, Cost_Air = cost_list_air)} ELSE {best_ih = best2_mmh.INVOKE ("Optimum", Goods = info_goods, … ... best2_mmh.TERMINATE() // Setup connections to megamodules. // Set global variables valid for all invocations // of this client. // Get information from the megaprogram user // about the goods to be transported and about // the two desired cities. // Get all routes between the two cities. //Get all city pairs in these routes. //Calculate the costs of all the routes. // Figure out the optimal megamodule for // picking the best route. //Pick the best route and display the result. // Terminate all invocations CHAIMS

  42. Data objects: Gentype bags Minimal Typing within CHAIMS: Integer, Boolean only for control All else is placed into an ASN.1 bag, transparent to compiler : A Gentype is a triple of name, type and value, where value is either a simple type or a list of other gentypes (i.e. a complex type). Simple types: given by ASN.1, the ASN.1-conversion library for C++, our own conversion routines. Example: Person_Information Name of Person complex Personal Data complex Address First Name string Joe Last Name string Smith Date of Birth date 6/21/54 Soc.Sec.No string 345-34-3456 CHAIMS

  43. Scheduling: Simple Example 1 cost_ground_ih = cost_mmh.INVOKE ("Cost_for_Ground", 1 List_of_City_Pairs = city_pairs,Goods = info_goods) 2 WHILE (cost_ground_ih.EXAMINE() != DONE) {} 3 cost_list_ground = cost_ground_ih.EXTRACT() 3 cost_air_ih = cost_mmh.INVOKE ("Cost_for_Air", 2 List_of_City_Pairs = city_pairs,Goods = info_good) 4WHILE (cost_air_ih.EXAMINE() != DONE) {} 4 cost_list_air = cost_air_ih.EXTRACT() order in unscheduled megaprogram order in automatically prescheduled megaprogram CHAIMS

More Related