1 / 34

Operationalizing SOA – Lessons Learned

Operationalizing SOA – Lessons Learned. Arun Majumdar (Cutter Consortium / VivoMind Intelligence Inc. ). Take Home Msg. : Multi-Level MDA & FOL. Agenda. SOA has 3 basic Stages for Operationalization re-engineering, data types (Ontology), process workflows

ermae
Download Presentation

Operationalizing SOA – Lessons Learned

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Operationalizing SOA – Lessons Learned Arun Majumdar (Cutter Consortium / VivoMind Intelligence Inc. ) Take Home Msg. : Multi-Level MDA & FOL

  2. Agenda • SOA has 3 basic Stages for Operationalization • re-engineering, data types (Ontology), process workflows • Apply Incremental development and Semantics • Data model, information transfer model, • Transformation model • workflow model • Lesson Learned: SSOA Meta-Model • An aspect-oriented architectural approach Real World Case Studies

  3. Case Studies • A large US Gov. Consulting Firm (80 offices across USA, UK, Japan) • Large TELCO (one of the big 5) • Big Pharma. Company – Semantic SOA – 200 Subsidiaries & many product lines Key Focus: Lessons Learned

  4. A Large Consulting Company SOA for Workflow Automation

  5. Desires • Wanted to have a new, better, faster, Contracting, Billing and Work delivery system. • Wanted to reduce or remove manual low level menial tasks in the corporate business processes • Use new technology for a web-based service façade. • Wanted to have hot-swappable services with new unforeseen capabilities add-ons • Remove Vendor Lock-Ins (Oracle, Siebel, BEA etc… ) and other grand dreams and desires … Desire: To improve the bottom line

  6. Client’s Situation #*%&! Gap: How to get there from here

  7. Client Situation – How it was Situation: Processes, Activities, Workflows

  8. Automation Tools: Semantic ETML Strategy: Extract Legacy, Transform & Model

  9. From Legacy to CG’s To Services • Focus on the transfer/transformation mechanisms. • Focus on Business Semantics and Ontologies using automation tools. • We built our own tools using Popkin, VBA, C-code, Prolog (which we called “LegacyWorks”, “Intellitext” for parsing data to Conceptual Graphs ) : • http://www.cutter.com/architecture/fulltext/reports/2002/09/index.html Lesson: A Business Ontology

  10. CICS CMSQL System System System CG  Transfer Architecture Observation: Businesses Do Lots of Transfers

  11. CG  Transformation Architecture Drill Down: Transfers with Transformations

  12. CG Business Process Architecture Fusion: A System of Transfers and Transforms

  13. Lessons • Need good tool-integration platforms for automation. • Model and develop adaptors and wrappers using Meta-Data (MD) • Choose a communications model to replicate the business processes via MD. • Abstract away all the specific low level functions in the system modules and focus on: • Transfers • Transformations • Develop these into “Services” Building: A Service Oriented Architecture

  14. A Large Telco SOA for Order Configuration Management

  15. Desires • Wanted to have a new, better, faster, Order Management System. • Wanted to avoid Manual Knowledge Extraction (error prone, costly process) • Use Semantic Web style technology for CLEC/ILEC/B2B/B2C … all interfaces • Wanted to have hot-swappable services configurations with new unforeseen capabilities add-ons • Remove Vendor Lock-Ins (Oracle, Siebel, BEA etc… ) and other grand dreams and desires … Desire: Improve the bottom line

  16. Desires Desire: An Elegant Dream!

  17. Analyze Existing Requirements ASR (Access Service Request) Order Type Partitioning Knowledge Extraction ASR Type Business Process and Rules Build Workflow Models Workflow and Data Models Generate Workflows Code Client Situation The initial BEA WLI (WebLogic Integration) test required 2-3 months for just one simple (New Connect) workflow; but, there are hundreds of workflows, hence years of work! This approach requires developers to have expert level knowledge of BEA’s WLI; not possible in the timeframe of the project so hire experts from BEA; but Telco training has yet to begin. Limited programming resources and domain expertise mismatches cause delays. Situation: Processes and Workflows

  18. Reality Reality: Telephony is very complicated!

  19. Reality Reality: This picture was developed in Visio™ !

  20. Step 1: Work with Real Data • Start with specific, measurable, in-use “data” itself: i.e.: Oracle DBMS’s • Refactor into from the vendor language, PL/SQL to a standards based language (UML). • Rebuild as Workflows. Inventory the Data: Data Semantics are key

  21. Step 2: Scan Source Code PACKAGE PK_LWIRE AS /******************************************************************************* Release : DEC03 Project Name : CAFE Module Name : Livewire Purpose : This package validates the Address in Livewire Tables Used : Global Variables : G_Lwire_mode Called from : Procedures Invoked : Nil Parameters Used : Modification History : -------------------------------------------------------------------------------- . . . G_Lwire_Mode VARCHAR2 ( 1 ); G_Ld3_changed VARCHAR2(1) := 'N'; PROCEDURE SP_LW_REQUEST_INIT; PROCEDURE SP_LW_REQUEST_MAN; PROCEDURE SP_LW_REQUEST_AUTO; PROCEDURE SP_LW_VALIDATE_ASR( P_Asr_Id IN NUMBER, P_Req_Type IN VARCHAR2, P_Ret_Code OUT NUMBER); PROCEDURE SP_LW_CHECK_OVERRIDE ( P_Asr_id IN NUMBER, P_Refnum IN VARCHAR2, P_Loc_ind IN VARCHAR2, P_Ovr_flag OUT VARCHAR2 ); PROCEDURE SP_LW_VALIDATE_ADDR ( P_Asr_id IN NUMBER , P_Refnum IN VARCHAR2, P_Loc_ind IN VARCHAR2, P_Req_type IN VARCHAR2, P_Ret_code OUT NUMBER ); /* Nov03:C383700 - Procedure added for supp candidacy check*/ PROCEDURE SP_SUPP_CAND ( P_Asr_Id IN NUMBER, P_refnum IN VARCHAR2, P_Loc_ind IN VARCHAR2, P_supp_Cand OUT VARCHAR2); . . . Code snippet from the Oracle package PK_LWIRE (entire package is several thousand lines of code.) Automation: without tools, its too hard to do!

  22. Step 3: Code to Prolog (Horn Clauses) struct_code(package,[plsql, ,END,PK_LWIRE,(;), ,PACKAGE,BODY,PK_LWIRE,AS, ,PROCEDURE,SP_LW_REQUEST_INIT,IS, ,CURSOR,Asr_Crsr,IS, ,SELECT,A,.,Asr_ID,Asr_Id,(,), ,A,.,Act,Act,(,), ,SUBSTR,(,A,.,Reqtyp,(,),1,(,),1,),Reqtyp,(,), ,B,.,Status,Status,(,), ,A,.,Supp_Pending,Sup_Flag, ,FROM,Cta_Asr,A,(,),Cta_Asr_Status,B, ,WHERE,A,.,Asr_Id,=,B,.,Asr_Id, ,AND,B,.,Flowmode,=,',I,', ,AND,B,.,Status,IN,(,4040,(,),7050,),(;), , ,L_Asr_Id,NUMBER,(,13,),(;), ,L_Ret_Code,NUMBER,(,3,),:,=,0,(;), ,L_Sqlerr,VARCHAR2,(,80,),:,=,NULL,(;), ,M_Sqlerrm,VARCHAR2,(,80,),(;), ,M_Sqlcode,NUMBER,(;), ,M_Cafe_ref,VARCHAR2,(,100,),(;), ,M_Err_Pt,NUMBER,(;), ,L_actl_count,NUMBER,:,=,0,(;),BEGIN, ,M_Err_Pt,:,=,10,(;), ,BEGIN, ,M_Err_Pt,:,=,20,(;), ,SELECT,DECODE,(,Lw_Mode,(,),0,(,),',N,',(,),1,(,),',Y,',(,),2,(,),',D,',(,),',N,',), ,INTO,G_Lwire_Mode, ,FROM,Ctr_Livewire_Control,(;), ,EXCEPTION, ,WHEN,No_Data_Found,THEN, ,L_Sqlerr,:,=,SUBSTR,(,Sqlerrm,(,),1,(,),50,),(;), ,INSERT,INTO,Cta_Intf_Log,(,Module,(,),Req_Id,(,),Status,(,),Message,), ,VALUES,(,',L,',(,),0,(,),',1,',(,),',No,Data,Found,In,Ctr_Livewire_Control,',),(;), ,G_Lwire_Mode,:,=,',N,',(;), ,WHEN,Others,THEN, ,M_Sqlcode,:,=,Sqlcode,(;), ,M_sqlerrm,:,=,SUBSTR,(,Sqlerrm,(,),1,(,),80,),(;), ,M_Cafe_Ref,:,=,',CONTROL,TYPE,',|,|,',LWIRE_MOD,',(;), ,SP_INS_SYS_MESSAGES,(,',PK_LWIRE,.,SP_LW_REQUEST_INIT,',(,),M_Cafe_Ref,(,), ,M_sqlcode,(,),M_sqlerrm,(,),',Y,',(,),0,(,),M_Err_Pt,),(;), ,L_Sqlerr,:,=,SUBSTR,(,Sqlerrm,(,),1,(,),50,),(;), ,INSERT,INTO,Cta_Intf_Log,(,Module,(,),Req_Id,(,),Status,(,),Message,), ,VALUES,(,',L,',(,),0,(,),',1,',(,),',Ctr_Livewire_Control_Error,',|,|,L_Sqlerr,),(;), ,G_Lwire_Mode,:,=,',N,',(;), ,END,(;), ,M_Err_Pt,:,=,30,(;), ,FOR,Asr_Rec,IN,Asr_Crsr, ,LOOP, ETC . . .]). [PACKAGE,PK_LWIRE,AS, , , ,G_Lwire_Mode,VARCHAR2,(,1,),(;), ,G_Ld3_changed,VARCHAR2,(,1,),:,=,',N,',(;), ,PROCEDURE,SP_LW_REQUEST_INIT,(;), ,PROCEDURE,SP_LW_REQUEST_MAN,(;), ,PROCEDURE,SP_LW_REQUEST_AUTO,(;), , ,PROCEDURE,SP_LW_VALIDATE_ASR,(, ,P_Asr_Id,IN,NUMBER,(,), ,P_Req_Type,IN,VARCHAR2,(,), ,P_Ret_Code,OUT,NUMBER,),(;), , ,PROCEDURE,SP_LW_CHECK_OVERRIDE,(, ,P_Asr_id,IN,NUMBER,(,), ,P_Refnum,IN,VARCHAR2,(,), ,P_Loc_ind,IN,VARCHAR2,(,), ,P_Ovr_flag,OUT,VARCHAR2,),(;), , ,PROCEDURE,SP_LW_VALIDATE_ADDR,(, ,P_Asr_id,IN,NUMBER,(,), ,P_Refnum,IN,VARCHAR2,(,), ,P_Loc_ind,IN,VARCHAR2,(,), ,P_Req_type,IN,VARCHAR2,(,), ,P_Ret_code,OUT,NUMBER,),(;), , , , ,PROCEDURE,SP_SUPP_CAND,(, ,P_Asr_Id,IN,NUMBER,(,), ,P_refnum,IN,VARCHAR2,(,), ,P_Loc_ind,IN,VARCHAR2,(,), ,P_supp_Cand,OUT,VARCHAR2,),(;), , ,PROCEDURE,SP_LW_CHECK_OTC,(, ,P_Asr_id,IN,NUMBER,(,), ,P_Refnum,IN,VARCHAR2,(,), ,P_Loc_ind,IN,VARCHAR2,(,), ,P_Req_type,IN,VARCHAR2,(,), ,P_Otc,OUT,VARCHAR2,(,), ,P_Ret_code,OUT,NUMBER,),(;), , ETC . . . Scanner output showing procedure tokens The Main Idea: It’s not about Prolog, but LOGIC

  23. Step 4: FOL to XMI <?xml version="1.0" encoding="windows-1252"?> <XMI xmi.version="1.1" xmlns:UML="omg.org/UML1.3" timestamp="01/23/04 11:14:13"> <XMI.header> <XMI.documentation> <XMI.exporter>Enterprise Architect</XMI.exporter> <XMI.exporterVersion>2.5</XMI.exporterVersion> </XMI.documentation> </XMI.header> <XMI.content> <UML:Model name="EA Model" xmi.id="MX_EAID_6DCAB6E5_74DB_4e03_AAA8_11417163F24B"> <UML:Namespace.ownedElement> <UML:Class name="EARootClass" xmi.id="EAID_11111111_5487_4080_A7F4_41526CB0AA00" isRoot="true" isLeaf="false" isAbstract="false"/> <UML:Package name="Activities" xmi.id="EAPK_6DCAB6E5_74DB_4e03_AAA8_11417163F24B" isRoot="true" isLeaf="false" isAbstract="false" visibility="public"> <UML:ModelElement.taggedValue> <UML:TaggedValue tag="created" value="2001-03-05 00:00:00"/> <UML:TaggedValue tag="modified" value="2001-03-05 00:00:00"/> <UML:TaggedValue tag="iscontrolled" value="FALSE"/> <UML:TaggedValue tag="lastloaddate" value="2004-01-22 20:46:45"/> <UML:TaggedValue tag="lastsavedate" value="2004-01-22 20:46:45"/> <UML:TaggedValue tag="isprotected" value="FALSE"/> <UML:TaggedValue tag="usedtd" value="FALSE"/> <UML:TaggedValue tag="logxml" value="FALSE"/> <UML:TaggedValue tag="batchsave" value="0"/> <UML:TaggedValue tag="batchload" value="0"/> <UML:TaggedValue tag="phase" value="1.0"/> <UML:TaggedValue tag="status" value="Proposed"/> <UML:TaggedValue tag="complexity" value="1"/> </UML:ModelElement.taggedValue> <UML:Namespace.ownedElement> <UML:ActivityModel xmi.id="EAID_6DCAB6E5_74DB_4e03_AAA8_11417163F24B_ActivityModel" context="EAPK_6DCAB6E5_74DB_4e03_AAA8_11417163F24B" name="ActivityModel" visibility="public"> <UML:StateMachine.transitions/> <UML:StateMachine.top> <UML:CompositeState xmi.id="EAID_6DCAB6E5_74DB_4e03_AAA8_11417163F24B_Activity_Top" name="{top}"> <UML:CompositeState.substate> name> SP_LW_RESPONSE_PARSE <UML:ActionState name="SP_LW_RESPONSE_PARSE" xmi.id="EAID_C7946CD7_A379_4a6f_A353_56FE1875609E" visibility="public" namespace="EAPK_6DCAB6E5_74DB_4e03_AAA8_11417163F24B"> <UML:ModelElement.taggedValue> ETC . . . Partial XMI output for package SP_LW_RESPONSE_PARSE This new representation is used for importing into a tool (Enterprise Architect™) FOL: First Order Logic – Vocabulary & Terms

  24. Step 5: Import/Generate XMI to UML Importing Model from XML ***** Root Package: Activities ***** Add State: SP_LW_RESPONSE_PARSE Add State: SP_LW_LOC_DSG_IN Add State: SP_LW_RETREIVE_DEST_BBID Add State: SP_LW_INSERT_PREM_REQ Add State: SP_LW_RESPONSE_PROCESS Add State: SP_LW_IREP_REQUEST_FORMAT ***** Resolving Classifiers ***** ***** Resolving Ownerss ***** ***** Resolve Package Links ***** ***** Add Extensions ****** ***** Add Diagrams ***** ***** Resolving DiagramIDs ***** ***** Resolving Association Classes ***** ***** Resolving Parameter Types ***** ***** Resolving Attribute Types ***** ***** Resolving Note links ***** Import Complete! The import creates the nodes and the diagram into the UML repository (package by package) Transformations: Use Logic to generate XMI

  25. Outcome • Need good tools based on LOGIC (Common Logic is in ISO process) • Develop Models and Meta-Data using FOL (now Common Logic) and languages (for now, MOF, UML) • Focus on understanding the Workflows of processes: • Their Transfers of Data/Information etc.. • The Transformations they accomplish • Develop these into “Services” Outcome: Use FOL principles to guide SOA

  26. Big-Pharma SSOA for Knowledge Based Interoperability

  27. Desire • To provide timely knowledge from sources to clients (physicians, hospitals, labs…etc) within federal mandates • To save lives as a direct consequence of network performance in delivering knowledge • To provide information assurance (IA) and Multi-Level Security (MLS) • To provide high-availability, high-performance, optimized Quality Of Service • To provide end-user personalized experience Desire: Save Human Lives & Bottom Lines

  28. Raritan-N.J video video Breers - Belgium Failover Capacity Pipeline-2 ROUTERS video Sydney - Australia Video Pipelines Filled video Physical Network - Heterogeneity Failover Capacity Pipieline-1 Subscribers (Client Users) Channel Type: Publish (TrainingVideo) Scenario – Synchronized Training Problem: Space & Time, Content Synchronicity

  29. Other App. Servers – XML Messaging Core Objects Router Browser Web Server Server XHTML? Personalization TCP/IP HTML Transact XML WorkFlow App.Server ASP text Client? ROUTERS CORBA / RMI … form CGI How Can it All Work in synch? Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf What? Legacy Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd fdsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf Sd fdsa as dfsa asdf asd asd asd asdfasd asd t aasfd as asdf Sd f dsa as dfsa asdf asd asd asd asdfasd asd t asdf asfd as asdf JAVA, .NET, DCOM…??? Client Situation #*!? Situation: Spaghetti-Code and Systems

  30. XML: MetaData A DTD would be defined for each big pharma Trusted Partner, Affiliate, Company and this would include the corporate ontology and taxonomy. XSL - Form Generation XML: DTD Types & Taxonomy XML: Templates (Rules, Processes) Xlink - Channels Browser Behaviours HTML-GUI/ODBC/HTTP Xlink allows elements to create and describe links using XML syntax to describe simple unidirectional hyperlinks of regular HTML; Additionally, the syntax supports Channels & Pipelines to Services (via a pointer reference mechanism). XML - Flow Logic (Browser Engine) Pipeline #1 Pipeline #3 Pipeline #2 DB Remote Transactions Processor Personalization Info Effort : Cross-Cutting Concerns Top-Level: Connect Aspects / Ontologies

  31. The Channel Pipeline gets channels with specified security access and data-transfer capacity to the Router component of the Browser Portlet (object). This permits complete structural dynamism to be determined on a performance/availability/failover basis --- the individual allocated channels are used by the Accessor Component to search or acquire information, while the Viewer Component uses its special channels for streaming data displays. Channel Types text Browser image XML audio The App.Server provides these components Pass the different channel types along a special channel type called the Channel Pipeline video forms Portlet Channel Pipeline ROUTER ACCESSOR Channels can be Public, Broadcast, Private & Secured VIEWER Personalize Information Streaming Audio Streaming Video Communication Model : Aspects Node Broker Mid-Level: Aspects Model = Channels

  32. DATA • INFORMATION • KNOWLEDGE • INTELLIGENCE • WIRELESS / PDA • ROUTERS • COMPUTERS • CONTROLLERS Delivery Access An SSOA Meta-Model Emerges! • TAXONOMIES • HIERARCHY TYPES • SYSTEM TYPES • CAPABILITIES • TASKS • RULES • PROCESSES • ORHESTRATION • HUMANS • ACTORS • SYSTEMS • ENTERPRISES Ontologies Workflows Agents Function MDA MOF Workflow Resources Single Sign-on QOS QOI IA Content-Containers Hardware-Network SSOA NODE SERVICES REF: http://www.cutter.com/content/architecture/fulltext/reports/2006/01/index.html

  33. Outcome • Smart JAVA node-based server nodes • Channels with priorities to Brokers that connect clients to servers (using SOAP Objects/XML) • Channel monitors provide metrics (for IA, QOS) • Channel profiles deliver categories of personalized experience (physician vs. nurse) • Currently implementable with current tools, and able to accommodate future evolution Take Home : Multi-Level MDA via FOL methods

  34. Thank You

More Related