Excellence Around the Globe: Test Factory Services for Professional Testing in a Factory

2. Excellence Around the GlobeTest Factory Services Your Name

3. Agenda Why care about quality? Professional testing in a factory? How do I to get there? Why HP? Who has done it?

4. Why care about quality?

5. Why care about quality? If all that can happen is a project delay… • Results: HP's ordering system became chaotic • $400M loss in Q3 revenue • Damaged reputation with customers • Three executives were dismissed • HP's shares dropped 15% on the news • What is the impact of a project delay? • Project impact • A four week delay for a 75 person project = $1.3M in cost • 75 people x 900/person/day x 20 days = $1.3M • Business impact • In September 2004, HP struggled with SAP consolidation project resulting in a three week delay for their supply-chain software.

6. Success of IT projects Source: CHAOS Report 2007, Standish Group International, Inc.

7. The gap between the specification... And the user accceptance test HP CONFIDENTIAL

8. The gap between improvements... And the retest HP CONFIDENTIAL

9. Understanding defects Introduction of defects • Majority of defects are introduced during the requirements and design phase! • Howevermajority of defects are actually detected during user acceptance testing and in production! Detection of defects Source: NIST 2002 RTI Project 7007.011 10 November 1, 2006 HP Confidential

10. Assessing cost of defects This industry average is used as a baseline to calculate the cost savings Industry References: 3 B. Boehm and V. Basili, "Software Defect Reduction Top 10 List," IEEE Computer, IEEE Computer Society, Vol. 34, No. 1, January 2001, pp. 135-137.

11. The HP Quality Methodology (STLC) Idea Business Requirement Design Implement Deployed Application Operation Requirement Verification Test preparation Developer Test Production Functional Test / E2E Test System Test User Acc. Test EffortandCost TraditionalCost Planned Cost time

12. Defect (ISTQB): A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. Inserted defect: Defect created (inserted) in a phase Contained defect: Defect found in the phase it originated Introduced defect: Defect not found in the phase it originated Detected defect: Finding of an Introduced defect Defect detection rate & cost

13. Profesional testing in a factory?

14. Most organizations Introducing the HP Quality Model Customer success roadmap defined Level 4 Quality Expert Level 3 Quality Savvy Level 2 Quality Conscious Level 1 Quality Initialising Centre of Excellence Level 0 Quality Agnostic Service Utility Centralized technology & personnel QA processes govern testing and quality initiatives QA is thought leader and has enterprise influence Product Utility Centralized technology Centralized service bureau for QA personnel Centralized methodology and best practices for QA procedures Centralised technology, licenses, admin and support QA process in place, but mainly in individual projects Project-based technology, processes, & personnel The majority of testing, if not all, is still manual No documented QA processes or recognition of the need for them Testing, if done at all, is manual Increasing Maturity 11.50 – 12.0010m 1 of 2

15. Testing in the Lifecycle - today IT Supplier Requirements Business Analyst Design Unit Test Implement Design Business User Test Execution Test Environment Production

16. Professional Testing IT Supplier Requirements Business Analyst Design Implement Unit Test Design Business User Quality Factory AcceptanceCriteria ManualTest Test environment Test execution Test definition Test dataspecification Test automation Production Roll Out

17. Test Factory Service Catalog (1)

18. Test Factory Services Catalog (2)

19. Test Factory Services Catalog (3)

20. How to get there?

21. Building a Test Factory Time 1-2 days EnvisioningWorkshop • Present HP as solution Partner • Agreement on milestones for engagement 10-20 days Build blueprint for first test factory line • Solution Blueprint & Business Case 30-100 days Set up first factory line and run one cycle Pilot installation >100 days Application QualityRoll-out Service Development

22. Sample Timeline Q308 Q408 Q109 Q209 Q309 Q409 Phase 1: Preparation Phase 2: Setup the first line Phase 3: Operate the first line Phase 4: Evaluate the results and adjust Phase 5: Setup the next lines

23. Providing real time status informationDashboards as Program “Cockpit” Application Management Dashboard - EriDoc Functional Test Status Demand / Project Status Performance Test Status

24. Controls Flexibility Visibility Controlling the FactoryConsolidated top-down planning & bottom-up execution Audit trail Workflow Notifications Escalations Governance structures Business rules HP PPM Center™ Budgeting/costing Project proposals Issues, risks, scope changes Time tracking Change management Testing issues (defects) Operational work Resource capacity analysis Service desk or 3rd party integrations Project scheduling

25. Why HP?

26. Quality Simplify Standardize Optimize Strategic quality methodology Market-leadingHP Software Global deliverycapabilities HP Quality Services tri-dimensional approach aligns business with IT Improve performance Reduce backlog Ensure quality 27 1/1/2020 HP CONFIDENTIAL

27. Simply with HP Quality Methodology Quality Simplify Standardize Optimize Strategic quality methodology Market-leadingHP Software Global deliverycapabilities • Supports the application lifecycle, requirement to deployment • Integrates with every development methodology • Supports reuse of existing artifacts • Incorporates the business • Bridges the communication gap between IT and business 28 1/1/2020 HP CONFIDENTIAL

28. Blueprint - Based on real world experience

29. Standardize with HP Software Tools - Quality Simplify Standardize Optimize Strategic quality methodology Market-leadingHP Software Global deliverycapabilities Market leading product (60% +) Requirements management, risk-based testing, and cycle management First real-time link between business requirements and quality in a single enterprise platform. Delivers risk-based release decisions and end-to-end traceability. Introduces dimension of time and release cycles for agile quality Web 2.0 Performance validation in a world of AJAX and FLEX Accelerated testing SAP “TAO” HP Quality Management Ecosystem™ 30 1/1/2020 HP CONFIDENTIAL

30. Quality Simplify Standardize Optimize Strategic quality methodology Market-leadingHP Software Global deliverycapabilities Optimize with HP Global Delivery • 400+ plus testing professionals across GDAS Centers • Skilled professionals: Test Managers, Test Architects, Test consultants, Test Leads, Manual Test Engineers, Automated Test Engineers and Performance Test engineers, etc… • 150+ certified professionals - CSTE, CSTM, CSTP ISTQB, Brain bench. etc… • Strong people retention & development practices 31 1/1/2020 HP CONFIDENTIAL

31. Qualification • Learning and Development Program (Grow@hp) • Certified Tester Foundation Level • Certified Tester Advanced Level (Test Management) • Certified Professional for Requirements Engineering • Certification Program

32. Who has done it?

33. Total Cost of Defect Detection Project Example I – Automative Industrie (Experience) Testing Addendum Average Defect Number in comparable Projects (Experience)Standard Software (SAP/Siebel) /high Complexity due to Functionality / Interfaces 34

34. KMDInformation Technology Services IT improvements Business Outcomes 35 1 January 2020