© 2006, iTKO, Inc. | All rights reserved.

1. Achieving 99.99% Qualityin SOA Applications –The “Headless” Dilemma John Michelsen StickyMinds WebinarNovember 7, 2006 © 2006, iTKO, Inc. | All rights reserved.

2. Agenda SOA development and QA trends Strategy/best practices - the Three C’s “Headless” Testing Fundamentals LISA Intro, and LISA WS-Testing Offer

3. Who are we? • 2006 - LISA 3 & 3.5 • 2004 – LISA 2.0 A History of Quality Leadership • 2003 – LISA 1.0 Released to Market • 2001 – LISA under development – (4) Patents Pending • 1999 – iTKO Incorporates (Privately Held) 2004 . 1999 . 2001. 2003 . 2005. 2006. • Entered the market to deliver an SOA testing solution, LISA • Current version 3.5 (2006) • Key value propositions: • Increase Test Coverage • Lower Cost per Test • Faster Discovery & Resolution Everyone should own quality™.

4. Platform Evolution Causes Testing Evolution

5. The Promise of SOA • Reduce integration cost • Through loosely-coupled forms of integration and industry standards • Increase asset reuse • Build new business workflows from existing Services to form composite applications • Increase business agility • Better control business process definition and management to meet customer needs • Reduce business risk • Governance, compliance, and risk reduction through increased business visibility and agility

6. …before SOA Standard enterprise apps BrowserUIs Client UIs Server (database) You had a standard delivery platform. And a standard infrastructure.

7. SOA promises a composite workflow Division 1 Transaction service Transactionprovider PartnerReseller PartnerReseller Your extendedenterprise Order management Business rules BI tools SOA was supposed to simplify extending workflows. Core app Division 2 Customer buyer Data warehouse Outsourced supplier Customer buyer

8. But heterogeneous technology is complex Standards and components are still evolving Composite apps have multiple owners It’s Continuously Changing Your Company Channel Partner Division 1 Transaction service FinancialsMainframe Legacy App Legacy Data .NET OrderingService SOAP objects ContentDatabase Legacy App Messagingservice Web interface Your AppWorkflow Customer company BI tools Outsourced firm BusinessRules ESB Division2 Data warehouse File System CRM Web App RMI objects Web Services

9. Why A New Strategy? • Few changes at the database level, but multiplies issues downstream • For every middle tier change, there are hundreds of front-end impacts possible at the UI layer • Limitless front end options • Any client, multiple customers • Mix of technologies – XML/RPC, Swing, AWT, JS, DOM, now AJAX • Services consume business logic Database/mainframe = 1 change Middle-tierlevel = 100x UI level =1000x

10. Best Practices: “Three C’s” of SOA Testing Complete Testing • Every heterogeneous layer of architecture • Test UI verify in system of record • Reuse functional test for performance testing Collaborative Testing • Test early before UIs are created • Not just dev, business analysts and QA should verify processes Continuously Test • Regress on existing functionality • Add testing of new services to existing testing workflows Complete > single component all technologies Collaborative > Dev/QA test silos everyone tests, every phase Continuous > Phased tests constant testing

11. Complete: SOA Testing across every layer ASP.NET Web UI SOAP/XML .NET WS Custom API J2EE RMI MQ Database Legacy App File System Swing Presentation layer CORBA BPM Process and services layer ESB Integration layer Data / Applications

12. Typical Trading System Example Database CORBA App RMI Ordering Swing App App Server EJB, WS,POJO 1 BusinessProcess AppA BusinessProcess AppB ESB Processing Web App 9 AdditionalApps Now & Future AdditionalApps Now & Future AdditionalApps Now & Future AdditionalApps Now & Future DB Data is correlated in here.

13. Collaborative: SOA Requires Agile Business Analysts outline a business process and ‘not implemented’ test cases Developers unit test and jump-start QA Complete Test Coverage Lower Cost per Test Faster Discovery & Resolution QA expands dev tests and create functional and load tests Support rapidly debug issues and communicates appropriately Production test their implementation and report issues to support

14. Collaborative Testing in an SOA environment Order Acquisition Check Credit Check Inv QA/BA Place Order BPM Validation Orchestration Tests Place Order Service 2 Continuous SOA Test Platform Service 1 Service 3 QA QE Service Validation Service 3 Dev/QE Component Tests

15. Continuous: SOA is never done Database Create UI Serviceretired Create UI New service Servicechanged Create UI New service Database New service • Project-based development had a “test phase” Deploy App Database Createcomponents Unit test Integration Create UI AcceptanceTest • SOA development is constantly evolving

16. Continuous: Minimize Unintended Consequences APP 2 APP 1 Registry MDM Transactions ? ? ? ESB ? ? InternalService LegacyApp PartnerService Database NEW ? ? ?

17. Continuous: Minimize Unintended Consequences APP 2 Continuous Testing APP 1 A data service is currently failing due to a change in the internal service Registry MDM Transactions ESB InternalService LegacyApp PartnerService Database

18. Continuous: SOA at 5-9’s ?????%functionalintegrity 100% SOA-aware Dev/QA Process Comprehensive Testing Tools Continuous Testing Service BusinessApplications Integrity 100% 99.999%application availability Application Infrastructure App Servers Transaction Monitors Clustered Services Scalability 100% 99.999%system uptime Physical Infrastructure Availability Fault Tolerant HW Diesel Power Redundant Bandwidth Business Investment Availability to the business is the product of all three

19. Continuous Levels: Build & Deploy Production Production Deployment Level Staging Pre-production test bed Continuous SOA Test Platform test bed QA/QE Build Level Dev build Dev build Development Component/Service Teams product component product component

20. So how do we defeat the “Headless” dilemma?

21. Requirements are King -- On-line Music Scenario • Use Case: (UC1) Customer with existing account does a search for ‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1-click purchase feature to buy. • Business: (BR1) We need a flexible mechanism to change these criteria at our whim. After discussion: for now partners get 20% of net proceeds on their product sold on the first $100,000 / mo, 10% thereafter. • PSR: (PSR1) Search can’t take any longer than 3 seconds for a user on a typical broadband connection. • Policy: Punt for now. This is security, interop standards, corp dev standards, etc. • Functional: (FR1) Search must include results from CDs, DVDs, Books, and Memorabilia, with CDs on the top (UC1). • Technical: (TR1) Create a search service that aggregates CD, DVD, Book, and Mem data (FR2). (TR2) Search service execution must be under 2 seconds (PSR1).

22. Service Level Guidelines • UC & BR: Validation mechanism • PSR: Load/performance test at the service level • FR: Likely 80% of your requirements can be nailed right here • TR: Dev test at the component level Service Validation Service 3 Dev/QE Component Tests UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

23. Orchestration Level Testing Guidelines • UC & BR: Not usually • PSR: Often the best way to prove PSR • FR: Invoke a behavior, validate outcomes • TR: Simulate unstable/unavailable services Orchestration Tests Place Order Service 2 Service 1 Service 3 QA QE UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

24. Composite/BPM Level Testing Guidelines Order Acquisition Check Credit Check Inv QA/BA Place Order • UC: Primary focus, performs a cause that we validate at the service/component level • BR: Many times same as UC, sometimes not relevant • PSR: A few PSR scenarios proved at this level, but continuous testing is the key here • FR & TR: Hopefully very few BPM Validation UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

25. Typical SOA Test Map -- Dramatically Simplified Content Books Mem DVD CD Billing AP Media Repo UC1: Search & Buy UC1 Presentation Layer Content Mgmnt Commission Rules Svc Order Mgmnt Search CRM Facade Offers & Specials UC1 Media Delivery UC1 Batch Feeds CRM System

26. Typical SOA Test Map -- Dramatically Simplified Content Books Mem DVD CD Billing AP Media Repo BR1: Commissions Presentation Layer Content Mgmnt Commission Rules Svc Order Mgmnt BR1* Search CRM Facade Offers & Specials BR1 Media Delivery Batch Feeds CRM System

27. Typical SOA Test Map -- Dramatically Simplified Content Books Mem DVD CD Billing AP Media Repo PSR1: 3 sec search PSR1 Presentation Layer Content Mgmnt Commission Rules Svc Order Mgmnt Search TR2* CRM Facade Offers & Specials Media Delivery Batch Feeds CRM System

28. Typical SOA Test Map -- Dramatically Simplified Content Books Mem DVD CD Billing AP Media Repo FR1: Search is aggregate, CDs top results Presentation Layer Content Mgmnt Commission Rules Svc Order Mgmnt Search FR1 CRM Facade Offers & Specials Media Delivery Batch Feeds CRM System

29. Typical SOA Test Map -- Dramatically Simplified Content Books Mem DVD CD Billing AP Media Repo TR1: Create search on 4 repos TR2: 2 sec search Presentation Layer Content Mgmnt Commission Rules Svc Order Mgmnt Search TR1 & 2 CRM Facade Offers & Specials TR1 Media Delivery Batch Feeds CRM System

30. The Three Dimensions of Quality maturity Complete > single component all technologies Collaborative > Dev/QA test silos everyone tests, every phase Continuous > Phased tests constant testing

31. LISA meets SOA Testing Challenges • LISA is Complete - Test heterogeneous SOA • LISA can test every tier of an application, regardless of location • LISA can test every service technology with one tool, one test • LISA’s framework approach creates bridges to legacy services that means ‘you can get there from here’ • LISA is Collaborative – Everyone owns quality • LISA shines at testing all components before UI even exists, as well as integrating with the team’s ALM lifecycle and process • Unit, regression, system, load, and monitoring in one tool • Leverages diverse skill sets in Dev, QA/QE, and BA roles • LISA is Continuous – Enables Agile SOA • Tests constantly evolving, interdependent systems during and after deployment • Traps for the “what-if” unintended system-wide consequences of making changes or corrections to components.

32. LISA WS-Testing $5,000 Off for next 50k users • LISA WS-Testing Edition is THE tool for Web Services testing • All our web services goodies are in there: • WSDL-based, point and click • WS-I Interop testing • Functional testing and service simulation • UI and Command Line execution

33. Thank you for your interest! • iTKO Web Site Resources • URL: http://www.itko.com or http://www.ws-testing.com • White papers • Support FAQs, Tech Notes, and Forums (once registered) • Download WS-Testing or start a larger evaluation • More from me • See above • Blog: http://itko.blogspot.com • Email: Start with info@itko.com; I’m john@itko.com ? ? ?