Quality is about testing early and testing often

1. Quality is about testing early and testing often Joe Apuzzo, Ngozi Nwana, Sweety Varghese Student/Faculty Research Day CSIS Pace University May 6th, 2005

2. Overview • Introduction • Overview • Relevance • Our methodology: Quality Attributes • Main Message to project teams • QA Walkthrough • Iterative development and testing • Unit Test case creation • Conclusions • Estimate of QA successfulness • Lessons Learned • Recommendations • Summary • Question session

3. Introduction • The CS615/616 is the capstone course for Pace University’s Masters in Computer Science curriculum. The focus of which is to develop soft skills as well as introducing the student to formal software engineering procedures. • To develop these real world skills, the students are broken up into teams, assigned a project and thus, responsible for delivering it on time. • The customers must accept the provided solution for the corresponding team to achieve full marks. • Our team was assigned the duty of being the Quality Assurance team. Our goal was to assist in delivery of high quality projects, through reviews, testing and the establishment of best practices.

4. Relevance “It is widely accepted that a "test-early, test-often" approach can greatly assist in the production of quality software … the design of software can impact on the ease of testing so testing must be in borne in mind from project conception through to completion.” The Serco consulting group

5. Team approach • Reviewed 2003 2004 QA team’s findings • Researched Quality Assurance best practices • Evaluated development environment • Focus on customer satisfaction as a measure of software quality

6. Our methodology: Quality Attributes • Clarity of user requirements • Who are the stakeholders and who are the users? Match up each requirement with each one. • What are the most important, least important, which have the greatest risks? • What are the user stories? Write them down! • Are there Conflicting, Confusing or Incomplete user requirements?

7. Our methodology: Quality Attributes • Verifiability • Define how the program knows when to continue and when to generate an error. • Each component needs to verify that it's input, output and data is correct. When components fail to do this bugs will be harder to locate, since bad data may be passed from module to module before being detected. • Use an integrated system log that is updated with state at each input or decision point.

8. Our methodology: Quality Attributes • Modifiability ( modularity or reusability) • Document how other developers who are not familiar with the project should proceed in making code change. Explain how to locate components, where to find there interdependences, where special cases are explained, etc.

9. Our methodology: Quality Attributes • Usability • Set the expected level of usability and document how you will communicate this to the stakeholders. • define who and how the system will be used. • What prior knowledge show a user have and if they should have specific skills that an average person would not. • All assumptions should be listed like "users will be accustomed to MS Windows and using a web browser" • For each customer delivery review usability and have sufficient buy in by all stakeholders.

10. Our methodology: Quality Attributes • Security • Define what the security requirements are for each user and or stakeholder. • Define what a security issue would be, and how it would be mitigated. • How will this software protect the users data and from whom? • Do the stakeholders agree with the security statements?

11. Main Message to project teams • Test early • Test often • Test enough • Designate (know your role) • Communicate • Iterate

12. Iterative development and testing Integration test Integrate findings

13. Unit test case creation There are 4 full paths, 3 decision points, 11 sections. All of which should be included in a unit test.

14. Estimate of QA successfulness • Reviewing designs early helped each team to clarified design trade-offs. • Setting dates for when things should be delivered helped focus each team to keeping on task. • Problems were found during testing and corrected.

15. Lessons learned • Applying Quality Assurance from the outside of each team was not as effective as we thought it would be. • Soft skills were as important as technical skills. • Technological barriers still have the biggest impact to a project. • There is not enough time to do project work.

16. Recommendations • Decentralize Quality Assurance, make one member of each team the QA focal point. • Mile stone and commitments need to be mapped out as early as possible. • Start as early as possible with the QA part of new projects, reusing or reworking the existing documents from previous Teams.

17. Summary • There is still work to be done. • There is opportunity to streamline many QA processes. • Communication between the teams and also between team members can be improved. • As in other studies, the more frequent and thorough unit testing combined with a frequent and aggressive unit / system testing will result in a higher quality product that will have the greatest chance of delivering on time.

18. Question Session • Question? • Comments? • Suggestions? Thank you for your time