A Practical Model for Program Test Management

1. Specialist Interest Groupin Software Testing A Practical Model forProgram Test Management Graham Thomas RCOG 8th March 2005

2. Abstract • This talk proposes a practical model for program test management, based on experience gained from working as a Program Test Manager on two programs; handling card authorisation in the financial sector and global customer service management in service delivery. • The talk is focussed on how to bring together the disparate streams of development and testing across large programs so that they can work together successfully and drive the program forward. It also addresses how to control quality when you no longer carry out development or testing activities. 2

3. Agenda • Definitions, responsibilities and attributes • Challenges that you face • A Practical Model for Program Test Management • Lessons learnt • Conclusions 3

4. Planning Analysis Design Preparation Execution Validation Reporting DefinitionTest Management “The direct management of testing activities” 4

5. DefinitionProgram Management • What is a Program? “A broad effort encompassing a number of projects and/or functional activities with a common purpose.”Association for Project Managers - Web site Definitions • What is Program Management? “The management of multiple streams of development activities (including testing), each with their own teams carrying out detailed planning, management and execution activities.” 5

6. DefinitionProgram Test Management “The management of multiple streams of testing, each with their own test management in place which is responsible for detailed planning, management and control activities.” 6

7. Responsibilities • PTMs are not responsible for managing the testing activities. That is what we have Test Managers for! • So, what are PTMs responsible for if they are not managing the testing? • Responsibilities vary from program to program: • Testing oversight • Analogous to Program Management • Looking for gaps across the program • Getting the development groups to work together with the testers • Defining and managing the program testing approach 7

8. Attributes of a good PTM • Honesty, integrity, experience, credibility • Pro-active, (dynamic - if you think rushing round helps without causing panic) • Politically aware and astute • Focussed on the program and organisational goals • Independence 8

9. Program Topographies 9

10. Program Topographies 10

11. Challenges • Implementing program-wide processes • Lowest common denominator • Have to fix the broken processes first • Imposition by the centre • Budgetary control • Resources • Communication • Geographies 11

12. 2077.5 points in 20 years 2003 Budget $418 million Relative performance 100% 30 Points in 20 years 2003 Budget $33 million Relative performance 95% Ferrari vs. Minardi 12

13. A Practical Model for PTM I would propose that PTM is a QA role and not a QC role. • Your level of involvement is in strategy, planning, dependency management and influencing • To report effectively you need to understand at the detail level how each of the test streams are progressing • You need to understand progress on the program to be able to determine the correct testing actions to take • You need to know that the development and testing process is working across the program 13

14. A Practical Model for PTM • Program testing strategy • Program test planning • Program level test processes • Quality assurance of development and testing Communication, Co-ordination and Co-operation 14

15. Program Testing Strategy • Addresses the integrated testing needs of the program • Aligns the development and testing lifecycles • Identifies the testing approach • Establishes the QA process required to support the testing approach • Specifies the measurement and metrics procedures • Introduces the program test management team Supports the program’s goals and objectives 15

16. Program Test Planning • Program planning techniques are required • Identify milestones and dependencies • Stage Entry and Stage Exit reviews • Intermediate checkpoints • Suspension criteria & resumption requirements 16

17. Program Level Test Processes • Release Management • Configuration Management • Environment Management • Defect Management • Change Management • Risk & Issue Management All applied at the Program level 17

18. ZYP QLEKF SUWRG XCIDB WOPR ZYP QLEKF XCIDB SUWRG WOPR SUWRG ZYP XCIDB QLEKF WOPR RED BLACK YELLOW BLUE RED GREEN YELLOW BLACK BLUE BLACK RED YELLOW GREEN BLUE GREEN The Stroop Effect 18

19. } QA QA QA QA } QA QA QA QA QA QA QA QA Develop System Test Integration Test Acceptance Test Quality Assurance • Communication • Common view • Program framework • Audit vs. Review QA QA 19

20. QA doesn’t equal perfection I cdnuolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg. The phaonmneal pweor of the hmuan mnid. Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer be in the rghit pclae. The rset can be a taotl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Amzanig huh? Yaeh, and I awlyas thought slpeling was ipmorantt 20

21. Lessons Learnt • Know your role in the program • Maintain independence • Manage expectations • A program centric view is often different and sometimes diametrically opposed to a project specific view. 21

22. Conclusion • How to bring together disparate streams of development and testing across large programs • How to control quality when we are no longer carrying out the development and testing activities • A practical model for Program Test Management 22

23. Test Management Forum 23

24. Contact Details Graham Thomas graham@badgerscroft.com +44 7973 387 853 www.badgerscroft.com 24