Susan Windsor Principal Gerrard Consulting Limited +44 (0) 7974 808604 susan@gerrardconsunting.com Web: gerrardconsultin

1. Test Assurance Coming of Age? Susan Windsor Principal Gerrard Consulting Limited +44 (0) 7974 808604 susan@gerrardconsunting.com Web: gerrardconsulting.com Intelligent Testing, Improvement and Assurance

2. Why are we here? When I first started talking about Test Assurance a few years ago, there were lots of blank looks but it’s become a hot topic now. Economic climate? I’ll share some of my experiences with you to illustrate potential benefits and pitfalls. Perhaps you’re interested in this as a career option, or maybe it could help your project or organisation? Intelligent Testing, Improvement and Assurance

3. Topics What is Test Assurance? Approach to Analysis Delivery Options Stakeholder Benefits Building your Development Plan Meeting your objectives Intelligent Testing, Improvement and Assurance

4. What is Test Assurance?Definition (1) • Three main styles • Internal function with multiple project scope • External function to provide independent assurance on thoroughness, completeness and quality of testing • External function to provide guidance on improved effectiveness (e.g. project in difficulty) Intelligent Testing, Improvement and Assurance

5. What is Test Assurance?Definition (2) • Key characteristics • Insurance policy; if everything always went well it wouldn’t be needed • Never includes delivery responsibility • Always relates to implementation • Demands direct route to senior stakeholders Intelligent Testing, Improvement and Assurance

6. What is Test Assurance?Definition (3) • Terms of reference for recent assignment • Troubled project • Can send you a copy • Example Job Description • Use now to explore scope • Use to identify career development path • Need to be credible and make a difference quickly to be accepted • Can send you a copy Intelligent Testing, Improvement and Assurance

7. What is Test Assurance?Activities (1 and 2 of 6) • Establishing the requirement • Meet your stakeholder and understand their goals for you • Start of project/programme or at any point for a project in trouble • Analysis • How will you deliver your goals? • What is the current status? • What are the typical hot spots? • Gaps, duplication, environments, contracts, staffing, etc. • How long have you got to demonstrate your value? • Could be an hour, a day or a week – won’t be longer • We’ll explore a model for analysis shortly Intelligent Testing, Improvement and Assurance

8. What is Test Assurance?Activities (3 and 4 of 6) • Building your Team • Sometimes you have to do this alone • Is there administration support available? • Do you need specialists to support you? If so ask for them immediately • Write a short terms of reference to include your requirements, plans and deliverables - get your stakeholder to sign it and circulate to everyone on the project • Understanding your impact • Most people will not understand why you are there • Everyone you need to talk to is busy and may also be under extreme stress • Some people will not tell you the whole truth and may mislead you • There is always organisational politics • Human nature to believe “we’ll catch up”, “lets fix this in-house” Intelligent Testing, Improvement and Assurance

9. What is Test Assurance?Activities (5 and 6 of 6) • Building relationships • Have a clear mandate that you can talk to anyone you want to that is related to the project/programme and senior people in the organisation • Ensure everyone knows who your sponsor is • Accept you may not be popular but you need co-operation to be successful • Closure • You will be asked for, and expected to give, your opinion on • The overall testing that’s been undertaken • Which issues remain outstanding and require action • How to make a better start next time Intelligent Testing, Improvement and Assurance

10. Approach to Analysis • Consulting Model • An assessment method designed to support formal reviews of testing effectiveness • But now has to be second nature, rapid analysis essential • It takes a long time to write a short letter! • Information Gathering • As a minimum read project start-up documentation; test strategy; test plans, etc. • Look for “out of scope” and “assumptions”. • Supplier Review • What is their brief; how are they managed; what are their objectives • Use a supplier risk assessment checklist to seek out priority areas • Typical Project Issues • Gaps and/or duplications in test activities • Environments; Test data • Skill levels; Staffing numbers • Levels of reporting • Role of Project Office Intelligent Testing, Improvement and Assurance

11. Basic Consulting Model "Where are we now ?" Improve Baseline Current Culture Identify Actions Methods Measurement Implement the change Assignment Skills Technology "Do it !" Organization Where do we want to go next? Plan the transition "How do we get there ?" Intelligence Testing, Improvement and Assurance

12. Expand dimensions to suit your task Does everyone on the team have a common understanding and commitment to the project goals What could inhibit these being achieved Is everyone using the same methods; do they understand the methods; are they appropriate for this project Are the controls and measurements sufficient and appropriate; is the timing appropriate; how does information change as it migrates; is it accurate Culture Measurement Methods Assignment Technology Skills Organisation Are there too many/few staff assigned; do they have the skills/competencies required; is the level of management support sufficient Is the technology appropriate; is it implemented and supported; what issues exist on the ground using it; How effective is the project organisation; are there any communications issues; is the meeting culture appropriate; are there activities not owned by anyone; does everyone understand their role Intelligence Testing, Improvement and Assurance

13. Identify Realistic RecommendationsThis is a Must Have! Remember your time line – 1 hour/day/week! Relate findings to the context of the project; include high level architecture diagrams and organisation charts and mark the “hot-spots” Group findings into a small number of areas or “hot-spots” Rank the findings in each area by priority; consider impact on key project goals (always include timescale); don’t forget financial implications For the top “3” in each area give specific solutions; ask for exactly what you need; give options where possible As a minimum you must be prepared to verify that the reporting and status provided is accurate Intelligent Testing, Improvement and Assurance

14. Delivery Options External Consultant or Internal Assurance Single project/programme or all current projects/programmes Initial baseline and/or review Recommended actions Ongoing assurance Intelligent Testing, Improvement and Assurance

15. Stakeholder Benefits Gains a project independent and balanced view of the status and quality of testing Builds confidence in the predictability of implementation Provides rapid, accurate and valuable information to base their decisions upon Provides a fast path to key issues for them to resolve Intelligent Testing, Improvement and Assurance

16. Building your Development Plan?Bit of homework! How could you develop your skills and competencies; can you identify specific ones to work on; where can you find support for this How can you get relevant experience; could you use the consulting model to review existing projects; can you brief senior managers on the value of the function; can you get in a specialist to work alongside and achieve skills transfer Finding a mentor or an expert; who do you know that can already do this; how can you learn from them Intelligent Testing, Improvement and Assurance

17. And Finally……. • Ask me for any support materials you’d like • Job Description • Terms of Reference • Supplier Risk Assessment • Examples of suitable reporting formats • Feel free to ask me any questions • Contact me on susan@gerrardconsulting.com Intelligent Testing, Improvement and Assurance