DONE-01: Applying Quality Assurance and Testing to Application Development

1. DONE-01: Applying Quality Assurance and Testing to Application Development Ademola Taiwo (AT) Empowerment Consultant

2. Agenda • Introduction • Quality Defined • Why Quality Assurance • Implementing QA Infrastructure • Ensuring Quality • Summary • Questions DONE-01 - Applying QA and Testing to App. Dev.

3. Achieving Quality Goal: Implement Quality A Quality product meets or exceeds customer expectations DONE-01 - Applying QA and Testing to App. Dev.

4. What is Quality? Quality is All about the Customer • Quality is • Based on customers’ perceptions of a product’s design and how well the design matches the original specifications. • The ability of a product and service to satisfy stated or implied needs. • Achieved by conforming to established requirements within an organization. http://www.asq.org/portal/page DONE-01 - Applying QA and Testing to App. Dev.

5. What is Quality Control Quality Control is NOT Quality Assurance • Control = An evaluation to indicate needed corrective responses; the act of guiding or the state of a process in which the variability is attributable to a constant system of chance causes. • Quality control = The operational techniques and activities used to fulfill requirements for quality. DONE-01 - Applying QA and Testing to App. Dev.

6. What is Quality Assurance? Quality Assurance is NOT Quality Control • Assurance = The act of giving confidence, the state of being certain, or the act of making certain. • Quality assurance = All the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence a product or service will fulfill requirements for quality DONE-01 - Applying QA and Testing to App. Dev.

7. Cost of Poor quality Internal Failure Cost Costs associated with defects found before the customer receives the product or service DONE-01 - Applying QA and Testing to App. Dev.

8. Cost of Poor quality External Failure Cost Costs associated with defects found after the customer receives the product or service DONE-01 - Applying QA and Testing to App. Dev.

9. Cost of Poor quality Appraisal Cost Costs incurred to determine the degree of conformance to quality requirements DONE-01 - Applying QA and Testing to App. Dev.

10. Cost of Poor quality Prevention Cost Costs incurred to keep failure and appraisal costs to a minimum DONE-01 - Applying QA and Testing to App. Dev.

11. Cost of Poor Quality Quality Variables Drive Priorities • Quality Variables • Defects • Time To Market • Features • Internal Cost / Affordability DONE-01 - Applying QA and Testing to App. Dev.

12. Why is QA Necessary? The Boehm Curve Requirements Code Release TIME “It takes 5 to 17hrs to fix bugs in released code, as compared to 1 to 1.5 hrs during code reviews and inspections” US National Institute for Standards and Technology, May 2002, Economic Impact of Inadequate Infrastructure of Software Testing. DONE-01 - Applying QA and Testing to App. Dev.

13. Cost of Poor Quality To the US Economy • The National Institute of Standards and Technology study shows that Software defects cost the U.S. economy some $59.5 billion a year. (2002) • http://www.nist.gov/public_affairs/releases/n02-10.htm DONE-01 - Applying QA and Testing to App. Dev.

14. Cost of Poor Quality Internal Cost to the Development Organization “On average developers make 100 – 150 errors every thousand lines. These statistics are different developer to developer and project to project and language to language.” If 10% of the errors produced are serious in a small application of 20,000 lines, then we will have 200 serious errors!! Watts. S. Humphrey. Multiyear Study by Software Engineering Institute (SEI) at Carnegie Mellon University DONE-01 - Applying QA and Testing to App. Dev.

15. Cost of Poor Quality External Failure Cost (To You) • Blue Cross Insurance, US – Sent $60 million in unwarranted and duplicate payments • Pepsi Promotion, Mexico – Instead of issuing 1 x $40,000 winning bottle, 800,000 were produced • Trader in London sold shares 100 times larger than he intended. Bank had to pay out millions to retrieve these shares. Validation requirements were not catered for DONE-01 - Applying QA and Testing to App. Dev.

16. Benefit of Good Quality To Everyone (the Economy) • NIST's estimated that about $22 billion of the sunk costs could be recovered through better software testing • http://www.nist.gov/public_affairs/releases/n02-10.htm DONE-01 - Applying QA and Testing to App. Dev.

17. What is Testing Testing is NOT Quality Assurance Testing is the act of gathering metrics in pursuit of Quality. It generates supporting data for assuring quality. Testing is just another source of Quality metrics DONE-01 - Applying QA and Testing to App. Dev.

18. Why Test? A relatively inexpensive way to gather metrics • “Software defects cost the U.S. economy almost $60 billion annually. 80% of the total development costs are consumed by developers identifying and correcting defects” National Institute of Standards and Technology (NIST). www.nist.gov/public_affairs/releases/n02-10.htm • “Cancelled projects due to quality issues in the US, cost $55 billion dollars” The Standish Group. CHAOS Chronicles v3.0 March 2003 DONE-01 - Applying QA and Testing to App. Dev.

19. Agenda • Introduction • Quality Defined • Why Quality Assurance • Implementing QA Infrastructure • Ensuring Quality • Summary • Questions DONE-01 - Applying QA and Testing to App. Dev.

20. Implementing QA Infrastructure NOT Test Implementation • Verification • Is it error free? • Does it do what was specified? • Validation • Is it valid? • Is this what you really, really want? • Risk Mitigation • Expect the Unexpected http://www.its.bldrdoc.gov/fs-1037/dir-029/_4291.htm DONE-01 - Applying QA and Testing to App. Dev.

21. Benefits of Quality Assurance Better Customer Experience at Reduced Cost • Reduced Time to Market • Lowered Risk • Reduced Rework • Increased Job Satisfaction • Serving as a Differentiator • Efficient Framework for Outsourcing “IT outsourcing will increase by 130 percent in the next 5 years to $23 billion in 2008” Information Week 22nd October 2003 DONE-01 - Applying QA and Testing to App. Dev.

22. Quality is Everyone’s Job! Quality - or Lack of - is Ubiquitous Performance Usability Reliability Quality Interoperability Maintainability Integrity Adaptability Efficiency DONE-01 - Applying QA and Testing to App. Dev.

23. Implementing Quality Assurance Ensuring Quality DONE-01 - Applying QA and Testing to App. Dev.

24. Integrate QA into your Software Development Life Cycle (SDLC) QA begins at the Requirements Gathering Stage • QA Process should begin with requirement gathering and continue through to the application release. • Quality improvements require discipline. People, processes and technology are only successful with proper follow through. • QA processes must be supported by technology to help troubleshoot and analyse problems. • Testing should be a function of lowering risk (risk-based testing). DONE-01 - Applying QA and Testing to App. Dev.

25. Measuring Quality • Understand what metrics you want to measure • Define and communicate • Clear defect severity rating • Quality encompasses more than tests and defects! Think about: • Customer satisfaction • Features • Platforms • Defects • Costs – internal and external • Time to Market • Process • Feature / Functionality Definition • Spec reviews • Code reviews • Test scenario reviews • Project plan reviews • Measure and allow for improvements throughout the process DONE-01 - Applying QA and Testing to App. Dev.

26. Keys to Achieving High Quality Quality Product must Meet or Exceed Customer Expectation • People • Process • Technology DONE-01 - Applying QA and Testing to App. Dev.

27. Key to Achieving High Quality People: Quality / Development Organization • Development & QA Managers: Peers • General Engineering Managers (GEM) • Monitors Critical Quality Goals • Cross Dept Communication • Maintains Product Release Calendar • Monitors & Improves Development Process DONE-01 - Applying QA and Testing to App. Dev.

28. Key to Achieving High Quality Process: Guidelines • Quality ownership: engineers and designers must think about how to test and validate • Design time requirements • Product feature design • Test scenario design • Quality needs to be a ‘lifestyle’ not a checkbox item! • Design quality into the product NOT • Test to see if it is a quality product DONE-01 - Applying QA and Testing to App. Dev.

29. QA in Development Methodologies Process: Guidelines • QA Approach is determined by the Software Development Life Cycle (SDLC) • There are 3 main SDLC Models: • Waterfall Business Objectives Requirements Specification Functional Specification Technical Specification Program Specification Acceptance Test Code DONE-01 - Applying QA and Testing to App. Dev.

30. Development Methodologies Cont…. Process: Guidelines • V Model • RAD Check Check Check Check DONE-01 - Applying QA and Testing to App. Dev.

31. Current View Point – Application Transformation Applying Quality Assurance and Testing to App. Dev. DONE-01 - Applying QA and Testing to App. Dev.

32. Key to Achieving High Quality Technology: Automated Data Gathering • Testing • Functional Testing • Boundary testing • Negative testing • Platform Testing • Multi-user Testing • Stress / Performance Testing DONE-01 - Applying QA and Testing to App. Dev.

33. Test Plans Technology: Automated Data Gathering • Define test scenarios in the design specs • Define test cases in the test plans and automation • Intent needs to be clear • Review all test cases • Include multiple test types • Define measurable milestones • Feature Freeze, Code Complete, Code Freeze • Don’t forget performance testing • 5X slower is a bug ! DONE-01 - Applying QA and Testing to App. Dev.

34. Data Gathering Technology: Test, Test and Test again • Three phase of testing • Phase 1 – Planning through Feature Freeze • Phase 2 – Feature Freeze through Code Freeze • Phase 3 – Code Freeze to customer delivery DONE-01 - Applying QA and Testing to App. Dev.

35. Summary Quality is Everyone’s Job • Quality is meeting customer expectation • Good quality is exceeding customer expectation • QA enhances IT effectiveness • Increase Customer Satisfaction • involves People, Processes and Technology • Immediate ROI • Lower cost / Increased revenue • Found Money • Start NOW Start Now DONE-01 - Applying QA and Testing to App. Dev.

36. Questions? DONE-01 - Applying QA and Testing to App. Dev.

37. Thank you for your time! DONE-01 - Applying QA and Testing to App. Dev.

38. You Can Begin NOW! DONE-01 - Applying QA and Testing to App. Dev.

39. DONE-01 - Applying QA and Testing to App. Dev.

40. Agenda • Introduction • Quality Defined • Why Quality Assurance • Implementing QA Infrastructure • Ensuring Quality • Summary • Questions DONE-01 - Applying QA and Testing to App. Dev.

41. Summary Quality Assurance • Quality is meeting and exceeding customer expectations • QA involves People, Processes and Technology • Immediate ROI DONE-01 - Applying QA and Testing to App. Dev.

42. The Quality Health Check • Quality Health Check Focuses on Four Key Area’s: • Software Testing • Software Configuration Management (SCM) • Incident Management • Process Management across QA DONE-01 - Applying QA and Testing to App. Dev.

43. Quality Matters Applying Quality Assurance and Testing to App. Dev. • Bullets: Arial Bold, size 28, sentence case • Second level: Arial Bold, size 26 • Third level: Arial Bold, size 24 • Maximum 7 bullets per slide • Maximum 7 words per bullet • Type in charts and graphics • Arial Bold, size 20 or above, initial caps • Source code in examples • ABSOLUTELY NO SMALLER THAN SIZE 20 • See sample slides DONE-01 - Applying QA and Testing to App. Dev.

44. Proposed Break Down Software Testing (IEEE 7925-2) Review and Highlight Improvements in Processes, Procedures and Strategies: • Incorporate Reviews – Walkthroughs, Informal Reviews, Technical Reviews and Inspections • Conduct a Risk Assessment's • Look at Functional (Black Box), Structural (White Box) and Non-Functional Testing in line with SDLC • Incorporate a Test Strategy • Incorporate a Test Plan (IEEE 829-1998) DONE-01 - Applying QA and Testing to App. Dev.

45. Proposed Break Down Software Configuration Management (SCM) • Auditing • Release Management • Build Management • Process Control • Investigate the Workflow DONE-01 - Applying QA and Testing to App. Dev.

46. Proposed Break Down Software Testing (IEEE 7925-2) Review and Highlight Improvements in Processes, Procedures and Strategies: • Incorporate Reviews – Walkthroughs, Informal Reviews, Technical Reviews and Inspections • Conduct a Risk Assessment's • Look at Functional (Black Box), Structural (White Box) and Non-Functional Testing in line with SDLC • Incorporate a Test Strategy • Incorporate a Test Plan (IEEE 829-1998) DONE-01 - Applying QA and Testing to App. Dev.

47. Implementing Test Plans • Test at API Level as well as User Level • Automate versus manual test • Allow for multiple phases of development & testing • Phase 1 – development of features, unit testing & execution of automated regression tests. • Phase 2 – regression testing of existing functionality, performance, platform, and stress testing • Phase 3 – Final regression testing DONE-01 - Applying QA and Testing to App. Dev.

48. Regression Testing (example) • Criteria to Ship • Automation Passing at 95% or greater ( understand the failures and evaluate the risk versus time to market) • Rerun of failed tests • Regression testing validates • No defects introduced • New features function • Weekend Automated Regression Tests • Execution on all platforms • 2 days to complete testing • Nightly Automated Regression Tests • Execution on all platforms • 1 night to complete testing DONE-01 - Applying QA and Testing to App. Dev.

49. Proposed Break Down • Software Configuration Management (SCM) • Auditing • Release Management • Build Management • Process Control • Investigate the Workflow DONE-01 - Applying QA and Testing to App. Dev.

50. Measuring Quality • Understand what metrics you want to measure • Define and communicate • Clear defect severity rating • Quality encompasses more than tests and defects! Think about: • Customer satisfaction • Features • Platforms • Defects • Costs – internal and external • Time to Market • Process • Feature / Functionality Definition • Spec reviews • Code reviews • Test scenario reviews • Project plan reviews • Measure and allow for improvements throughout the process DONE-01 - Applying QA and Testing to App. Dev.