310 likes | 512 Views
INNOV-11: An Introduction to DataDirect Shadow Products. Brian Anderson. Director of Product Strategy, DataDirect. Agenda. Shadow/RTE – A Unified Platform for Mainframe Integration. History of DataDirect’s Shadow/RTE Mainframe Primer Accidental Mainframe Integration Architectures
E N D
INNOV-11: An Introduction to DataDirect Shadow Products Brian Anderson Director of Product Strategy, DataDirect
Agenda Shadow/RTE – A Unified Platform for Mainframe Integration • History of DataDirect’s Shadow/RTE • Mainframe Primer • Accidental Mainframe Integration Architectures • Mainframe Operational Challenges • Technical Architecture • Customer Use Cases • Questions
History of DataDirect’s Shadow Products • Progress acquired NEON Systems in Jan. 2006 • NEON Systems was leading mainframe integration vendor • Data • Events • Services • NEON Systems was a market consolidator in mainframe integration space • Acquired InnerAccess Technologies in July 2004 • Acquired ClientSoft in December 2004 • Products in existence back to 1987
Leader in Gartner MQ NEON Systems
Gartner Group on Mainframe Integration ““The future of the programmatic integration market is one of fewer vendors, more focused on SOA or event-driven architectures. They are the leading strategies selected by organizations as part of their legacy modernization initiatives...”” Dale Vecchio, Research VP, Gartner Magic Quadrant for Programmatic Integration Servers, 2005 Gartner Research, February 25, 2005
What is a Mainframe Proprietary hardware Proprietary OS Data throughput High availability Perceived Expensive Legacy code Backward compatibility Mainframe Manufacturers Stratus Bull Siemens ICL Unisys Hitachi Tandem (HP) Fujitsu IBM iSeries (formerly AS/400) z/Series VM VSE z/OS Mainframe Primer Introduction
Evolution of the IBM Mainframe from 1965 z/Series • z/OS • 64bit z/Series • z/OS • 64bit ES/9000 • OS/390 • 31bit 3090 • MVS/ESA • 31bit 3084Q • MVS/XA • 31bit 3033 • MVS/SP3 • 24 bit s/370 • MVS • 24 bit s/360 • MFT
State of the Art • IBM z/Series 2094/754 • $22m • Known as z/9 • 17802 MIPS across multiple z/OS images • 11687 MIPS in any single z/OS image • 54 General Purpose CPUs • 512GB main memory • 1024 I/O Channel Processors • 10gb/s Data Transfer rate • Java Co-Processor (zAAP) • Information Management Co-Processor (zIIP) • Crypto Co-Processor • Usage • 14000 z/Series machines worldwide • 50% z/OS (50% VSE, OS/390, VM, z/Linux)
Subsystems Data Flat Files QSAM,VSAM Hierachical IMS/DB, Adabas, Datacom, IDMS, M204 Relational DB2, Supra, IDMS/R, Oracle TP Monitors (App Servers) IBM CICS, IMS/TM Other IDMS/DC, Com-Plete Languages 2GL Assembler 3GL Cobol, PL/1, Fortran 4GL Natural, Mantis, IDEAL, CSP Batch/Online Business Applications (25-30 yeas old) Inventory management Credit card authorization Retail banking Risk management Parts tracking ATM Billing Supply chain General ledger Student registrations Logistics Payroll Mainframe Subsystems & Application Portfolio
Mainframe Integration Drivers • Mainframe Relevance • Greater than 60% of the world “system of record” data on mainframe • More than 450 of Fortune 500 rely upon mainframes • More commercial transactions processed on mainframe than any other platform • Business Pressures • Regulatory Compliance • Real-time Enterprise • Increased Agility • Total Economic Impact • Technological Initiatives • Business Intelligence • Business Process Optimization • Business Activity Monitoring • SOA & EDA
Typical Mainframe Accidental Architecture Replication Server CICS Integration IBM DB2 WORF IBM DB2 Connect Enterprise Edition SQL Access Gateway IBM CICS Transaction Gateway Oracle Transparent Gateway TIBCO/webMethods SeeBeyond Adapters 3270 Screen Scraping Management consoles
Significant Operational Issues • Quality of Service (QoS) • End-to-end solutions as strong as the weakest link • Fragile implementations often based on gateways • Lack of consolidated diagnostics • Latency due to increased complexity • Agility • Complexity slows support for new applications • Additional hardware deployment often required • Lack of clarity on which option to choose – analysis paralysis • Cost • Redundant hardware gateways for resilience • Software licensing fees for multiple products • Multiple vendor management • Training costs for administration of multiple product • Additional employee costs for effective management of all products • Cost of systems outage due to QoS issues
Typical Mainframe Accidental Architecture IBM DB2 WORF SQL Access Gateway IBM CICS Transaction Gateway Oracle Transparent Gateway IBM DB2 Connect Enterprise Edition Replication Server CICS Integration 3270 Screen Scraping Unified Management Console Shadow RTE – A Single Unified Platform Wouldn’t it Be Nice? Management consoles
Scalability Sysplex routing and load balancing WLM support Proven to support thousands of transactions per second Security SAF interface for all interactions Optimizations for session-less integration SSL encryption of conversation Client SSL certificate based authentication Audit/chargeback Transactions Exposes XA protocol to JTS via Shadow JDBC/J2CA adapters Server Distributed Syncpoint Resource Manager (SDSRM) Full Resource Recovery Services (RRS) integration In Doubt transaction management Reliable messaging Support for WebSphere/MQ transport for SOAP additions in w/MQ 6 RFH2 header support for URI Supports ErrorQ processing In conjunction with SDSRM enables Async Transactional Messaging for Web Services SC34-6651-00 (May 2005) details the implementation NEON implementation using MQMD for service/action and MQSTR structure Instrumentation Extensive. Low-overhead event tracing for all mainframe integrated workloads All nodes in the Sysplex log to the coupling facility Central correlation point for Sysplex transactions Eclipse viewing tool Quality of Service
Shadow RTE Pluggable Features • z/Direct:SQL access to mainframe resources Data and Applications via ODBC, JDBC or J2CA (JCA). • z/Services: Mainframe Web services (Publish and Consume) • z/Events: Real-time capture, enrichment and publishing of critical mainframe business events. • z/Presentation: Mainframe Web enablement • Shadow Studio:Full featured,Eclipse-based IDE
Shadow z/Direct • Direct SQL/Data access to: • DB2, IDMS/DB, IMS/DB, VSAM and Adabas • Supports programs using industry standard ODBC, JDBC or J2CA (JCA) without any intermediate hardware gateways • Makes mainframe applications appear as database stored procedures, supporting: • CICS, IMS/TM, Natural and IDMS/DC programs and presentation layer • Scales to thousands of transactions per second • Supports two-phase commit (2PC) for data integrity
Shadow z/Services • Web services - SOAP-based mainframe integration • SLI, BLI, DLI • Sources - CICS, IMS, IDEAL, IDMS, Natural, databases • Supports both Publish and Consumption • Simplifies mainframe integration • Flexible deployments • Mainframe • Shadow RTE • CICS • Distributed
SOA – Mainframe Perspective • Specific Relevance to Mainframe SOA • Arcane systems • Original tightly coupled development • “Fancy” programming for technical expediency of a bygone era • Existing mainframe integration methods force tight coupling • Web Services • Not a solution to organizational loose coupling • Without SOA, Web Services are just 21st century RPCs Web Services with SOA Web Services without SOA <SOAP-ENV:Body xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01"> <s0:QSTAFF> <s0:QSTAFFInput> <s0:ID xsi:type="xsd:short">10</s0:ID> </s0:QSTAFFInput> </s0:QSTAFF> </SOAP-ENV:Body> <SOAP-ENV:Body xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01"> <s0:QSTAFF> <s0:QSTAFFInput> <s0:ID xsi:type="xsd:short">10</s0:ID> <s0:FLAG xsi:type="xsd:char">X</s0:FLAG> <s0:ACCODE xsi:type="xsd:short">8</s0:ACCODE> <s0:STAT xsi:type="xsd:short">5</s0:STAT> </s0:QSTAFFInput> </s0:QSTAFF> </SOAP-ENV:Body>
Shadow z/Events • Mainframe Business Event Management • Real-time Change Data Capture • Maintain data consistency with mainframe system of record • Automate heterogeneous business processes • Non-invasive event capture • ADABAS, DB2, Native VSAM, CICS VSAM, IDMS and IMS DB • Messaging support • WMQ, HTTP/HTTPS, as well as inbound J2CA communications
Shadow z/Presentation • Mainframe screen access via Web browser • Development of “web” channels • Access mainframe screen-based applications • Repurpose/Rejuvenation user interface • Screen aggregration for process reengineering • Non-invasive • Integrate with other Web components
Success Story - ABN/AMROShadow/RTE • Why DataDirect? • Shadow RTE selected through exhaustive RFI/RFP process • Shadow RTE achieved “technical win” during POC • ABN/AMRO advised that early metrics show >50% development time reduction and a 30% reduction in mainframe MIPS • Shadow RTE enables ABN/AMRO systems to achieve application agility – escaping the limitations of their previous technologies • PERFORMANCE PERFORMANCE PERFORMANCE! • Challenges • Due to compliance issues, ABN’s commercial banking unit required a replacement of MDp • Essentially all business applications are embedded into MDp, MQ, Gideon and Sybase servers, including mortgage, teller, commercial loans, small business, VRU • Over 1M request/day flow through Gideon • Wanted to eliminate need for MDp developer and reduce development time by 50% • More importantly, they required a solution that would preserve and use much of the existing business logic and client code
Success Story - Her Majesty’s Land Registry (UK) – z/Direct • Results • Shadow provided enterprise-class integration for business-critical, distributed applications with the company’s DB2 mainframe database. • Faster, reliable access to DB2 data resulted in higher quality of service. • Performance metrics exceed 30,000,000 SQL calls per hour (peak), 300,000,000 SQL calls per day – every day • Reduced management complexity • Challenges • Needed an easy-to-use, industrial-strength mechanism to deliver access to mainframe data in a mission-critical application from a Java environment • Required support for both ODBC and JDBC applications accessing DB2 without a middle-tier • Performance was an absolute requirement • 4th largest DB2 shop in the world
Success Story - Sempra UtilitiesShadow z/Services • Why DataDirect? • Deal signed on 9/29/05 • Initially whittled vendors down to InnerAccess and ClientSoft in 2004 • Decision suspended for a few months • Sempra called us back in April 2005 to move forward with ClientSoft due to superior development environment • When customer found out ClientSoft had merged with NEON, they were extremely pleased • Sempra views Shadow RTE as a truly strategic offering – a unified platform for mainframe integration, not just a tactical Web services generator • Challenges • Facilitate development of mainframe Web services for eServices customer portal (self-service) and new business contractor • Key problem was sheer complexity and time requirements of hand-coding of mainframe Web services. • No mainframe-based in-house Web services talent
Success Story – Wells Fargo • Results? • Customer since 1996 • Leveraged Shadow RTE to create customer portal driven primarily from existing mainframe functionality • Contains an assortment of Web services for servicing customer accounts through a variety of channels e.g. internet banking, customer services, etc. • Accesses primarily legacy data and transactions in DB2 and VSAM/CICS environment via XML, SOAP, and proprietary messaging. • Over 7mm invocations of mainframe Web services per day • Challenges • Open up existing legacy databases and applications • Facilitate development of mainframe customer portal • Expand channels to service customer • No mainframe-based in-house Web services talent
Success Story - Miami-Dade County, Florida • Results • Standardizing access to core mainframe functionality across 40 county departments • Easily incorporated mainframe into new J2EE applications via Web services • Reduced IT development and maintenance costs moving forward • Enhanced services to county employees and taxpayers • Challenges • Needed to create a service-oriented architecture that would standardize access to multiple disparate mainframe systems • In an effort to build new Web & wireless applications, needed to enable effective reuse of mainframe functionality via Web services to build new apps • Wanted to eliminate need for expensive point-to-point integration
Success Story - Electric Insurance Company • Results • Projected revenue increases of 30% by enabling more efficient pricing changes • Eliminated need for redundant COBOL rating engine • Enabled mainframe to consume .NET Web service from more modern Web-based rating engine • By incorporating mainframe into new SOA, now have ability to offer umbrella coverage nationwide via the Internet • Challenges • Needed to enhance revenues by bringing pricing changes to market in a more efficient manner • Wanted to eliminate the need for redundant policy rating functionality • Needed to enable the mainframe to directly consume rating data from an external, Web-based rating engine via Web services • Required interoperability within Microsoft .NET
For More Information, go to… • PSDN • www.shadowrte.com • www.datadirect.com A complete series of detailed product webinars including demonstrations are available at the above URLs.