World Quality Report 2012-13 Testing for the Future

1. World Quality Report 2012-13Testing for the Future Presentation to NOSQAANovember 15, 2012

2. Company Overview

3. Sogeti Testing Practice Overview

4. Introductions Dan Hannigan is a Senior Vice President with Sogeti USA LLC. Sogeti is  a wholly owned subsidiary of Capgemini SA. Dan has been with Sogeti for 24 years, during this time he has held several positions including: Director of Consulting Services specializing in package implementations, Manager of Systems Professionals, Project Manager, National Practice Manager for Seibel Implementations, Account Executive (sales), Geographic Vice President for the Akron/Cleveland offices and is currently the North American Practice Manager for the Sogeti Quality Assurance and Testing Practice (Managed Testing Practice). His current responsibilities include supporting Sogeti Geographic units in the sales and delivery of QA/Testing solutions, development of solutions within the practice and Global responsibility to the Sogeti Global team for Strategic development. Dan also sits on the Global Innovation Board which is responsible for evaluation of potential Innovations for inclusion in the Sogeti QA/Testing Solutions. Prior to joining Sogeti Dan was the CIO for the  Conna Corporation Convenience Stores chain a division of Dairy Mart Convenience Stores.

5. Agenda

6. What is the World Quality Report? • Only global report for application quality covering: • Outsourcing trends, tools, technology, organization • Industries: CPRD; E&U; FS; High-Tech; Public; TME • Geographic analysis: ANZ; Brazil; China; France; Germany; Netherlands; North America; Nordics; UK • Co-sponsored report by Capgemini, Sogeti & HP • Capgemini Group (world leader in testing services) • HP (World leader in application testing technologies • Research sample and methodology • 1,553 telephone interviews with CIOs, CFOs,IT directors and senior managers involved intesting and QA based on a structured questionnaire. • Fourth year the report has been published

7. World Quality Report: Interviews by Region • WQR research based on 1,553 telephone interviews covering 25 countries • Data from questionnaire and customer quotes

8. World Quality Report: Interviews by Sector and Job Title Interviews by Sector Interviews by Job Title

9. 2012-13 World Quality Report Details 6 x 2 page sector analysis 9 x 2 page country analysis,top trends

10. North America • Respondents data • Number of respondents: 296 • Type of companies: Medium and Large Private Sector companies, Government and Public Sector Organizations, • Breakdown as Follows: Based on FTE Headcount 10,000+ (119), 5,000-9,999 (99), 1,000-4999 (78) • Telephone interviews carried out April – May 2012 • Research conducted by specialist B2B market research company, analysis done by our own specialists in the Industry • Other information on country/sector • 58% of NA Companies plan to use Testing-As-A-Service in the next 12 Months • 44% of NA Companies say budgets have increased with 50% saying additional increase by 2015

11. North America • Other information on country/sector • The Cloud is seen as a way to optimize cost management, improve flexibility and help align IT with the needs of the business. • Only 20% of NA organizations rate internal testing resources as above average in terms of quality • 35% of organizations report that the testing function is highly decentralized and a further 41% operate combined centralized and decentralized teams • 64% of companies report that internal Testing functions are performed by business analysts or development teams. • 30% of companies plan to implement an internally managed TCOE in the next 2 years • 19% of organizations intend to use 3rd party TCOE capability

12. North America • Other information on country/sector • 1 in 10 organizations is still implementing a TCOE started in 2011, but only a minority have made the full transition. • The commitment to testing is demonstrated by the allocation of IT budget to testing, at 18% it is on par with the global average, but more encouraging is that 40% of this budget is allocated to innovation/new transformational projects. • 44% of organizations report that the IT budget is increasing and half are reporting that they expected continued increases thru 2015. • 6% of organizations state clearly that the budget is either static or decreasing, not particular sector was identified. • 60% or organizations reported that the greatest increase in investment is in tools and outsourcing

13. Key Findings and theirimplications for Organizations

14. QA budgets: Stepping up Majority of organizations expect testing budgets to rise between now and 2015.

15. QA budget: Stepping up • General and universal trend over regions and sectors • 42% reported their testing budget has increased last year  same level of investment in quality is maintained: show importance of QA . • 53% expect their QA budget to rise between now and 2015 • Indicate optimism for future • Testing spend on: • 41% new transformational work (testing new application) • 59% maintenance work (maintenance of existing software)

16. QA budgets: Stepping up My investment going forward will [focus on] bringing my internal teams up to standard, increasing the level of automation used, and developing greater efficiencies throughout. Although our testing budget is not separate from the wider IT budget, the level of monitoring and performance measuring has increased – we need to justify our costs. – An automotive business, Brazil “ • The problem is increasing pressure to get things done yesterday. We have this conflict between testing adequately to our satisfaction and the testing time expected by those on the business side. Investment has increased but then expectation in speed of delivery has also increased. It ’s not easy, as the organization needs to be able to jump into a much faster testing mode while maintaining quality standards. • – A Manufacturing Business, CZECH REPUBLIC “

17. QA budgets: Stepping up Figure 3 Areas of Greatest Increase in Spend

18. QA Budget: Stepping up • Investment priorities: • Budgetary line item: upgrade existing process and tools Why? Move to newer application lifecycle platforms and software-on-demand models • 2015: 46% of businesses will dedicate their budget to purchase external resources to supplement their internal QA team. • Investment priorities vary by region: • Eastern Europe invests more in new testing tool (33% compared to 19% generally and 15% for North America) • Public sector leads the way for increased QA budget:  rise of 48% compared to the previous year despite budget cuts.

19. Testing Centers of Excellence: A growing priority Organizations are turning to COEs to consolidate their QA operations in response to market pressures, technology demands, and emerging models of testing delivery.

20. Testing Centers of Excellence: A growing priority • Market unpredictability: some businesses still looking to shave 30-40% off their testing budgets. While QA budgets are increasing, still a need to control costs • Technology expectations: growing demand to deliver applications as quickly, efficiently and as cheaply as possible • Diversity of delivery models: QA departments are maximising on efficiency gains offered by the cloud, and agile testing (currently being used by 37% of testers)

21. Testing Centers of Excellence: A growing priority A TCOE is something I’m building here; it represents a significant change in our approach but one that I personally believe to be vital to our business since I’ve come from an organization with an operating testing center. We know that it can reduce costs and overall time of testing practices, and given the competitive nature of the market we operate in, it’s critical to get speed and costs down while getting quality up. We believe a TCOE can do this. “ • – A MANUFACTURING BUSINESS, UK • For us, the move towards a TCOE is part of the wider union between the business and IT side of the organization. We now have a much stronger link between the two, and while that means we feel more able to understand and answer demands, it does increase pressure on testing to be more efficient as a whole – decentralized testing teams just couldn’t provide that efficiency and it meant that walls existed between the teams, whereas shared experience and best practice are far easier within a TCOE framework. • – A financial services business, Finland “

22. Testing Centers of Excellence: A growing priority Figure 3 Expected Benefits from Implementing a Testing Center of Excellence

23. Testing Centers of Excellence: A growing priority • Why build a TCoE and what benefits to expect? • Speed to market: a key of growing TCoE adoption • Compressing time-to-market (36%) • Enhance the agility of QA operations (17%) • To cope with challenges of an unpredictable economy: • Produce better and faster • Competition and maximize profits • Contribute to enhance quality and better resource management: • An efficient approach to industrialize testing activity • Provide visibility into the level of quality for any software system or project • A centralized pool of resources that can be leveraged across multiple projects in an organization

24. Only 31% of firms surveyed currently test mobile applications. Why? According to many interviewed for the report they have no appropriate tools or expertise and limited access to the devices to test effectively mobile applications. Mobile Testing: Behind the curve

25. Mobile Testing: Behind the curve • Why such a need of mobile testing? • Proliferation of devices • Need for mobility: performance and usability on the move • Increase in demands and expectations  Shift to mobile devices: more than 5 billion mobile devices in the world for only 2 billion computers Quote: “More and more of our business is done online [in general] so it’s something [mobile app] we need to get hold of and understand quickly” A retail business, UK Real need of mobile testing which is not yet fulfilled....why?

26. Mobile Testing: Behind the curve • Companies are underprepared: • Organizations are ill equipped: • 65% do not have the right tools • 52% do not have access to required devices • Fragmentation of mobile devices and frequent releases:  a very changing market which requires adaptability from testing organizations

27. Mobile Testing: Behind the curve • Average: 31% of organizations test mobile application • Up to average: • Regions with traditionally high levels of mobile application (Australia & NZ, Nordics, Western and Southern Europe) • Asian challenges: • Explosion of the mobile market in Asia • Only 30% of organizations test mobile application so far • Focus on portability and compatibility across devices

28. Mobile Testing: By Sectors • Manufacturing firms are surprisingly more likely to conduct mobile testing • Some sectors have difficulties to adopt mobile testing (Public sector, Financial services...)

29. Mobile Testing: By sectors Quote on page 28 Retail Business UK  increasing investments are expected over the coming year

30. Mobile Testing: Priorities when testing mobile applications

31. Mobile Testing: Priorities when testing mobile applications • Why testing mobile applications? • Performance: increase efficiency is a priority for 64% of respondents • More acute in the UK & Ireland: 75% set performance as priority demonstrating the maturity of this market • Security: only 18% set security as a priority on average • Suggest changing perspectives in security concerns (Bring-your-on-device culture and protection of data)

32. Cloud: Gathering Pace • 28% of testing currently occurs in the cloud environment • This is set to rise to 39% by 2015 • Only 4% reported no plans for Cloud testing, down from 31%, 2 years ago.

33. Cloud: Gathering Pace The cloud has become a significant priority for our organizational direction in the Information Technology Department. With regards to testing, it’s opening up a way of turning on and off a stream of new skills and abilities, that is both fast and cost-effective. – A financial services business, The Netherlands “ We experience peaks and troughs in our testing program, so TaaS provides a certain degree of flexibility. While some Applications may be run of the mill, some may have more ad hoc requirements. TaaS allows us to meet unique or temporary needs without the burden of great cost or adding to project time. – A retail business, Denmark “

34. Cloud: Rapid Growth In the UK the Cloud has more than gathered pace: Only 11% don’t plan to use the Cloud.

35. Cloud: Benefits Organizations are rapidly turning to the Cloud for cost savings, flexibility, agility better resource and time management.

36. Quality Assurance Resources: A question of value 59%: qualify their internal QA teams as on “average” Only 4-5% are confident that their resources (internal and external) are best in class

37. Quality Assurance Resources: A question of value • Critical to assess the quality of software applications for organizations  Need confidence in the quality of QA resources  In developed countries, QA is becoming a recognized and dedicated profession • 33% of companies express more confidence in outsourced resources than in their own employees  desire to increase QA skills • Find the right balance between in-house and external teams

38. Quality Assurance Resources: A question of value Outsourced testing will always be an important option, but how we source those testers does tend to depend on project or cost, rather than any specific predefined rules. We used to use India-based suppliers a lot more than we do today, but the cost benefits were not balanced by our internal client satisfaction feedback. – A financial services business, Australia “ I’d love to be able to professionalize our testing function – give people a career and the pay to match and they’ll stick with you. But the reality is quite the opposite. Testing is seen as a ‘necessary evil’ in many organizations. This means Organizations never give [testers] the proper attention [they] deserve and we end up buying external experts who may have the skills, but not necessarily the intimate knowledge of our business. – A retail business, Denmark “

39. Quality Assurance Resources: A question of value • North America: most highly outsourced region for QA (to lower-cost labor market) • FS: recognize much more than any other sector the value of experienced QA resources

40. Key findings by sectors

41. Consumer products, Retail and Distribution • Today’s trends: mobile and internet commerce • Invest in QA/testing to meet higher expectations from customers • 41% say that their QA budget has increased (35% previously) • Need of innovation : 45% of testing activities implemented in new or transformational projects • Our survey findings: • TCoEs: 6% already functional, 25% plan to develop them • Cloud: 21% of their applications are cloud based • Mobile : 34% are currently testing mobile application. The key criterion in this domain is the capacity to test application on several networks (60%)

42. Energy & Utilities • Undergo major structural transformation (liberalization of markets) • Challenge: maintain antiquated legacy systems and invest in innovative product lines  IT budget has remained largely static this year (for 51% of respondents) • Our survey findings: • TCoEs: a large proportion say having no plans for a TCoE (44%) • Cloud: 1 in 5 of their applications hosted in the cloud • TaaS: 47% plan to use TaaS within next year • Mobile: 65% do not test mobile application • Preference for upgrading internal expertise

43. High Tech • Fiercely competitive sector with a quick pace of change • Evolving relationship and interaction between consumers and organisations • 41% say that their IT budget has increased this year • Plan to invest in new testing tools by 2015 (45%), much more than any other sector  This sector will lead the way of innovation, what is reflected in its approach of QA • Our survey findings: • TCoE: 43% plan to develop TCoE • Cloud: 62% of their applications are hosted in the cloud • TaaS: 13% use this service and 43% intend to use it within next year • Mobile: 30% currently test mobile application