Trends in the Marketplace Testers will have to change – but how?

1. Trends in the MarketplaceTesters will have to change – but how? Paul GerrardGerrard Consulting1 Old Forge CloseMaidenheadBerkshireSL6 2RD UK e: paul@gerrardconsulting.comw: http://gerrardconsulting.comt: 01628 639173 Assurance with Intelligence

2. Agenda • It’s the Benefits, Stupid! • Automation frameworks • Test Process Improvement is a Waste of Time • Software Success Improvement • What Makes a Good Tester? • Recommendations Assurance with Intelligence

3. It’s the Benefits, Stupid Assurance with Intelligence

4. Why projects fail • Won’t bore you with yet another survey of IT projects that fail - take it for granted, most do • But why? • We can trace most failures to: • Bad decisions • Decisions that were made too late • Decisions that were not made at all • We suggest this happens because stakeholders and project managers lack the right information at the right time • We’ll call this information Project Intelligence The frame of reference for making those decisions is often wrong too! Assurance with Intelligence

5. Four-eyed plans Increasingly Inaccurate IT- focused Initial Plans The plan is a model of the project. The real project consists of people, organisation, goals and risks. Assurance with Intelligence

6. Team Poster • The people like to see the broad range of issues covered by the politicians • Makes politicians feel important • But it’s all froth benefits It’s the economy, stupid! Ultimately, our customers are only interested in the benefits of IT Assurance with Intelligence

7. Gartner predictions • Number of people in IT will drop by 15% by 2010 • 60% percent of technology professionals will move to more business-focused roles, concentrating on the use of IT and processes rather than IT delivery • In 2010, the typical IT department in a large company will be at least one third smaller than it was in 2000 • Departments within business will take on the traditional roles of IT. Assurance with Intelligence

8. Here’s my interpretation… • Increasingly, businesses will focus on benefits and will take control of projects or IT completely • Projects involving IT will no longer be dominated or managed by IT • The disciplines of Benefits Realisation, Goal-Directed Project Management and Project Intelligence will become mainstream • Most testers will work for (or come from?) business • (Some) Test Managers become PI Managers. Assurance with Intelligence

9. I am very grateful to Susan Windsor (susan@wmhl.co.uk) for the use of some of her material Automation Frameworks Assurance with Intelligence

10. Testing is in demand, and solutions reduce the number of functional testers required • Automation frameworks and more complex business requirements • Agile development methods mean developers undertake more unit and component testing • Growth of outsourced testing to different geographies => greater competition for the roles Is the role of the functional tester (who is neither technical nor business specialist) dead? Assurance with Intelligence

11. Functional test automation is broken! • Focus on technology rather than business needs • 80% of functional testing still manual • 60% to 70% of automation tools used for non-functional testing • Typically, traditional functional automation stops at 100 scripts, regardless of test coverage requirement • Critical factors • Cost of implementation and maintenance prohibitive • Insufficient and expensive skills required • Inability to asset share over different technologies Source: Paul Herzlich, OVUM UK Assurance with Intelligence

12. Business analysts already using them, and use will grow • Home grown frameworks built within organisations to meet business demands • Niche suppliers provide frameworks - try Google: • 34,400 exact matches for “test automation framework” • A few are now mature • Latest review by Paul Herzlich (OVUM analyst) • Market Leaders such as Mercury developing Business Process Tester (BPT) This seems to be the tools industry direction now Assurance with Intelligence

13. Governance Requirements Analysis & Design Project Management Test Management Test Case Definition – Automation Framework Test Case Documentation Manual Test Execution & Documentation Non UI Component Test Execution Harness UI Automated Test Execution Tools Test Requirements Sign Off Test Results Where frameworks fit Assurance with Intelligence

14. Test Process Improvement is a Waste of Time Assurance with Intelligence

15. How to improve… • I want to improve my (insert any activity here) • _______ people improvement • _______ organisation improvement • _______ process improvement  Changing people (like me) and organisation (like my company) is so hard – let’s not even think about it Assurance with Intelligence

16. The delusion of ‘best practice’ • There are no “practice” Olympics to determine the best • There is no consensus about which practices are best, unless consensus means “people I respect also say they like it” • There are practices that are more likely to be considered good and useful than others, within a certain community and assuming a certain context • Good practice is not a matter of popularity. It’s a matter of skill and context. Derived from “No Best Practices”, James Bach, www.satisfice.com Assurance with Intelligence

17. The delusion of process models(e.g. CMM) • Google search • “CMM” – 12,100,000 • “CMM Training” – 12,200 • “CMM improves quality” – 4 • A recent client… • CMM level 3 and proud of it (chaotic, hero culture) • Hired us to assess their overall s/w process and make recommendations (quality, time to deliver is slipping) • 40+ recommendations, only 7 adopted – they couldn’t change • How on earth did they get through the CMM 3 audit? Assurance with Intelligence

18. Assurance with Intelligence

19. But process models make improvement simple don’t they? • People like simple models: • levels of maturity, stepping stones, checklists, roadmaps and outside support for credibility • But life is much more complicated, unfortunately “Things should be made as simple as possible, but no simpler” Albert Einstein Assurance with Intelligence

20. People need process? • A big problem with process is it becomes all encompassing • Process folk sell process and cast all things in terms of it • They ignore that people who are smart • Smart people succeed regardless of process, not because of it • It could be argued, that less smart people need process • (By less smart, we're talking about people who need so much structure and enforced discipline they can only operate in the military, or in prison probably) • Is our industry really staffed by such people? • Do we really want production-line workers? • Do YOU really want to be a production-line worker? Assurance with Intelligence

21. Physics quotes… “I believe that a scientist looking at nonscientific problems is just as dumb as the next guy” “It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong” Richard P. Feynman Assurance with Intelligence

22. You can quote me if you want… “I believe that a process consultant looking at non-process problems is just as dumb as the next guy” “It doesn't matter how beautiful your process model is, it doesn't matter how smart you are. If it doesn't agree with reality, it's wrong” “Test process improvement is tinkering” Paul Gerrard ;-) Assurance with Intelligence

23. Software Success Improvement Assurance with Intelligence

24. ProcessChange From this… Perceived Results Chain CurrentMaturity FutureMaturity Acts of Faith CurrentCapability FutureCapability Better Capability = better, faster, cheaper Assurance with Intelligence

25. Changed Aspirations To this… Actual Results Chain CurrentConstraints/ Problems Acts of Change CurrentCapability FutureCapability Better Capability = better, faster, cheaper Assurance with Intelligence

26. Constraints, problems and aspirations • Constraints are fixed headcount, budget, timescales, quality of requirements, contracts etc. • Problems are “testing takes too long; too expensive; can’t hire testers; bugs get through” etc. etc.… • Aspirations: • Personal: personal development, fulfilment, motivation • Organisational: hero culture to team culture, outsourced, higher consistency, predictability • Acts of change are… Assurance with Intelligence

27. Acts of change – focused on constraints, problems and aspirations • Changes in behaviour to address specific problems (time, cost, quality etc.) • Targeted personal and team development • Infrastructure change (process, techniques, tools, environments) to support the changes • Managed Transition… Assurance with Intelligence

28. Principles of change • Current behaviour assessed in the context of current problems, goals and constraints • Aspirations drive the definition of goals • People in the job define and consent to the required changes in behaviour • People supported by • Personal/team development plans • Infrastructure investment (process, technology, tools, environment) specific to the change • Transitions are managed, not assumed. Assurance with Intelligence

29. Eight stage change process (Kotter) • Establish a sense of urgency • Create a guiding coalition • Develop a vision and strategy • Communicate the change vision • Empower broad-based action • Generate short term wins • Consolidate gains, produce more change • Anchor new approaches in the culture. Derived from “Leading Change”, John Kotter, www.johnkotter.com Assurance with Intelligence

30. Eight stage change process (after Kotter) Changes identified here • Mission • Coalition • Vision • Communication • Action • Wins • Consolidation • Anchoring This is where your ‘process model’ comes into play Assurance with Intelligence

31. What Makes a Good Tester? Assurance with Intelligence

32. A perfect tester? Pedantic Sceptic Nitpicker Assurance with Intelligence

33. Attributes of a good tester? redoubtable imaginative curious stubborn thick-skinned pedantic accurate observant assertive articulate thorough sceptical systematic intelligent logical conscientious persistent Assurance with Intelligence

34. Attributes of a good tester? Personal intelligence and skill Personality to cope with your environment redoubtable imaginative curious stubborn thick-skinned pedantic accurate observant assertive articulate thorough sceptical systematic intelligent logical conscientious persistent Assurance with Intelligence

35. Tester selection criteria • Intelligence - are they smart enough? • Thinking skills • Approach to problem solving; ability to reason • Interpersonal skills • Communication, assertiveness, conflict handling, survival, team skills etc. etc. • Testing skills • Theory (ISEB/ISTQB Etc.) • Practical (hands on testing) • Technical (technology, business domain etc.) • Plus… Personality • Many well-known attributes are usually required. Assurance with Intelligence

36. Testing theory (ISEB/ISTQB etc.) see www.bcs.org.uk/iseb www.istqb.org Thinking skills verbal reasoning, numerical/abstractreasoning, fault diagnosis, accuracy Testing Practice (Testing Case Studies) hands-on practical test activity Interpersonal skills communication, assertiveness,conflict handling, survival, team skills Tester skills (and training budget?) 20% 20% 40% 20% Assurance with Intelligence

37. Summary and Discussion Assurance with Intelligence

38. It’s the benefits, Stupid! • Take a closer look at benefits realization, results-based management, performance management, project intelligence • Become more business-oriented • Be ready to talk their language – benefits and risk • Align your test activities with the need to provide information on benefits and risk to them • Be prepared to be managed by your customer! Assurance with Intelligence

39. Automation frameworks • Take a close look at frameworks • What is your organisation doing about it? • Understand how Frameworks support testing so you have a view, because you will be asked • If you are functional tester – evolve or… • Look to enhance your skills… • Become an automation/framework specialist • Move towards management • Take on non-functional skills • Get closer to your business/customers Assurance with Intelligence

40. Software Success Improvement • Don’t take on Test Process Improvement projects • Too many problems in test are caused elsewhere • Testers shouldn’t compensate for other people’s failures • Whole-process is in scope for change - or nothing! • Ask why things are the way they are • Focus on people, culture and organisation • “The art of the possible” – get your suggestions from your practitioners, not a book • Follow the 8 step change process (or another transition management method – there are many) • Be realistic about benefits, and be prepared to measure them Assurance with Intelligence

41. Developing, as a tester • Functional testing skills are a commodity – so move on • Have a vision of where you want to be… manager, specialist, consultant, business expert… • Following technology changes can be lucrative, but is a never-ending journey • Broaden your horizon – thinking, interpersonal, business, management, project management skills • Create a development plan – training yes, but look out for practical hands-on, not theory • Most learning takes place on the job so choose jobs wisely • Find a good coach. Assurance with Intelligence

42. Trends in the MarketplaceTesters will have to change – but how? Good luck in your career! gerrardconsulting.comuktmf.comriskbasedtesting.com Assurance with Intelligence