Improving Software Testing by Observing Process

1. Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan and Karo Mazidzhyan

2. Breakdown • Introduction • Related Research • Research Process • Analysis Results • Process Improvement Propositions • Conclusion

3. Introduction • The objective of this study was to understand how software testing is conducted by observation. • From observations propose improvement to the testing process. • Improvements by reducing development and testing costs, and improving quality.

4. Software Costs and Quality • Software Engineering strives to reduce development costs and improving quality. • Software Process Improvements (SPI) are the means to reaching these goals. • Commitment to SPI’s by all from all organizational levels is key to success • Quality can be tested into products or developed and built into products

5. Software Costs and Quality Cont. • External events such as deadlines affect software quality. • The cost of software testing is high, therefore SPI’s are necessary to reduce cost.

6. Related Research • Involvement of testing during development occurs when testers develop test for developers to analyze • The complexity of testing increases as a function of the complexity of the systems under testing. • Testing strategy defines the contents of testing.

7. Related Research Cont. • Communication and interaction between development and testing processes requires cooperation and coordination. • The use of software components are increasing rapidly • Design outsourcing and distributed development increase the use of components. • Cost of Quality is “Free”, but being late with products may be more costly than fixing faults.

8. Research Process • This study consisted of Organizational Units (OU) which develop and test technical software for automation or telecommunication in Finland. • Initial Sample included 26 OU’s, from which 5 were used as case studies. • Cases were chosen to show polar types

9. Research Process Cont. • Data for the research was collected by a series interviews. • Each interview had a different theme in mind and possibly a different interviewee in mind. • The interviews took place during five rounds, based on the theme.

10. Research Process Cont. • Interview Rounds • Development and Testing Managers were asked to define their testing process . • Managers of Testing were asked to define their testing process in depth. • Testers were interviewed. • Systems Analysts interviewed.

11. Research Process Cont. Case Breakdowns

12. Research Process Cont. • Data Analysis • Information gathered from these interviews were then categorized. • Categories were then analyzed to see how they were connected. • The categories were then used to identify factors which affected testing.

13. Analysis Results • Description of Cases: • Case A - Developed and tested Manufacturing Execution Systems : • Turnover 50% product, 50% service • Services included systems integration and customization • Testing against requirements was a challenge because customers had special in-house requirements standards • Developers and testers worked physically close to each other • Time allocated to testing was consistent, although over time it has been reduced • Use of components low, hinders testing

14. Analysis Results Cont. • Case B - Tested in house products and provided testing services for external customers: • Turnover 75% service, 25% product • Majority of requirements specifications were based on standards. • Delays in development allowed for fewer time allocated for testing • Communication flexible, developers talked face-to-face with testers • Use of components high, testability of components must be considered

15. Analysis Results Cont. • Case C – Customized Software Development: • Turnover 2/3 service, 1/3 product • Testers were involved early, involved in development process • Testers often had issues due to lack of advisement from developers • Delays in development does not often result in reduction of testing time • Developers and testers communicate face-to-face • Use of components low • Testing of software components seen as difficult b/c of different implementation.

16. Analysis Results Cont. • Case D - Electronics: • Turnover 100% product • High product orientation required high quality because recalls are very expensive • Avoided testing of unfinished product • Testing tasks clear and well documented • Testing involved in development late but planning of testing automation provided information on upcoming tests • Communication between developers and tester planned, formal and transparent • Use of components high • Components tested initially by suppliers then again at system testing.

17. Analysis Results Cont. • Case E – Software Testing Services: • Turnover 100% service • Working as an external testing organization required the adaptation of the process of the customer • Early involvement of testing was necessary for the testing company to increase the testability of the software • Sometimes testing was involved late • Budgets for testing affected testing time • Communication was handle through a contact person, but was active and clear • Use of components depends on customer • As an external testing organization it was hard to receive information about customers purchased components.

18. Analysis Results Cont.

19. Analysis Results Cont. • Cause and effect

20. Analysis Result Cont. • Cause and Effect • Business Orientation • Directly associated with use of components and testing schedules • Business Model – value adding process • Purely Service Oriented • System integration • Customizing • Consulting • Customers directly affect development and testing process • Purely Product Oriented • Product Development • Marketing • Customers do not directly affect development and testing process

21. Analysis Results Cont. • Process Improvement Propositions • Testing ought to be adjusted to business orientation • Product oriented should adopt formal planned testing process • Service oriented should adopt a flexible testing process • Enhanced Testability of software • Consider testability when selecting components • Review testing process of suppliers

22. Analysis Results Cont. • Effective Communication and interaction between development and testing • Early involvement of testing and planning of testing • Use of risk based testing

23. Conclusion • Proposals by observing best practices using grounded theory • Better documentation improved testability of software and components • Efficient communication between development and testing improved quality • When time is a major issue, risk based testing is the best solution • Business orientation affects: • Use of components • Amount and quality of communication • Allocated testing time and the planning of testing

24. Questions?