Quality Assurance in Sakai Past, Present and Future

1. Quality Assurance in SakaiPast, Present and Future Pete Peterson, Sakai Foundation Megan May, Indiana University

2. Agenda • What is QA? • Past • Present • Future • Discussion/Questions? 10th Sakai Conference - Boston, MA, U.S.A.

3. What is QA/QC? • Quality Assurance (QA) is the process of verifying or determining whether products or services meet or exceed customer expectations. QA considers all aspects of design, development, production, and service. When implemented correctly, QA provides a continuous and consistent improvement of processes within a product lifecycle. • Quality Control (QC), a subset of QA, is employed to ensure a certain level of quality in a product or service. The basic goal of QC is to ensure that the products, services, or processes provided meet specific requirements. • Often, Quality Control is confused with Quality Assurance. Though the two are very similar, there are some basic differences. QC is concerned with the product, while QA is process–oriented. Simply put, QA ensures that a product or service is developed and implemented in the right way; while QC evaluates whether or not the end result is satisfactory. 10th Sakai Conference - Boston, MA, U.S.A.

4. What is QA/QC? There are three essential questions that need to be answered and tested long before a product is placed into production, and QA/QC activities focus on answering these questions: • Does it do what we expect it to do? • Does it do what the end-user expects it to do? • Does it do “a” and “b” correctly and consistently? 10th Sakai Conference - Boston, MA, U.S.A.

5. Key Points of QA/QC • Quality is not free: There are costs associated with implementing these additional steps into your projects and development processes—time, money, resources, etc. These costs, however, are small in comparison to the benefits of having increased end-user satisfaction, reduced support requirements, and functionally accepted applications and systems. • Repeatable: The QA/QC process should be repeatable and flexible, so it can be easily applied to other projects as they arise. 10th Sakai Conference - Boston, MA, U.S.A.

6. Key Points of QA/QC • Comprehensive: QA/QC should be included in the project from the beginning to the end. The goal of this participation is to examine the project from all angles. This not only allows the QA/QC group to create better tests, but also to provide feedback and a global perspective to the developer community. 10th Sakai Conference - Boston, MA, U.S.A.

7. Key Points of QA/QC • Ongoing: QA/QC doesn’t stop once a project is complete; in fact this is where much of the QA is applied. Examining pitfalls and past mistakes can improve design, development and testing, as can documenting procedural successes and areas that need improvement. 10th Sakai Conference - Boston, MA, U.S.A.

8. Key Points of QA/QC • Autonomous: It is essential for the QA/QC group to have a high level of autonomy. Often the QA/QC process is performed in-house by the developers or project managers themselves. Although well-intentioned, this approach is almost always ill-advised and precarious. It is difficult to remove oneself, step back and see things from the diverse perspectives needed for an effective QA/QC analysis. 10th Sakai Conference - Boston, MA, U.S.A.

9. Past “Those who don't know the [past] are destined to repeat it.” Edmund Burke 10th Sakai Conference - Boston, MA, U.S.A.

10. Sakai’s Testing & Release Management Process

11. Formulation of QA Network Amsterdam Boston Cambridge Cape Town Columbia Indiana rSmart Charles Sturt Georgia Tech Recent additions of Nagoya, Marist & UMich

12. Past QA Workgroup Participation 2.2.1 23 People 10 Institutions 4 Countries 2.3.0 44 People 17 Institutions 5 Countries 2.4.0 93 People 37 Institutions 11 Countries 2.5.0 56 People 17 Institutions 4 Countries

13. Redefining processes • Defined milestones • http://confluence.sakaiproject.org/x/VwAiAQ • No new features in maintenance releases • Better oversight of merges into maintenance release branches 10th Sakai Conference - Boston, MA, U.S.A.

14. Other Accomplishments • Expansion of test script library • Raised awareness of the importance of unit and performance testing throughout community • Availability of Static Code review • Integration of Accessibility WG activities in general QA cycle • Inclusion of 3rd party licensing review in QA process 10th Sakai Conference - Boston, MA, U.S.A.

15. Sakai QA - Present 10th Sakai Conference - Boston, MA, U.S.A.

16. Sakai QA - Present • Finalizing Sakai 2.6 release • Finalizing Sakai 2.5 release • So everything is done right? 10th Sakai Conference - Boston, MA, U.S.A.

17. Sakai QA - Present • Sakai 2.6.1 and 2.5.6 coming soon to a QA server near you • Testing for maintenance branch merges • Updates of test plan 10th Sakai Conference - Boston, MA, U.S.A.

18. Sakai QA - Present The Links Slide • http://www.sakaiproject.org/portal • http://jira.sakaiproject.org/ • http://confluence.sakaiproject.org/ • QAhttp://confluence.sakaiproject.org/display/QA/Home • Release Managementhttp://confluence.sakaiproject.org/display/REL/Release+Management+Working+Group 10th Sakai Conference - Boston, MA, U.S.A.

19. Resources K2 Sakai 3 Sakai 2.5 K1 Sakai 2.6 Sakai 2.7 10th Sakai Conference - Boston, MA, U.S.A.

20. Future • Automation • Inclusion of test cases in every jira • Continuous testing 10th Sakai Conference - Boston, MA, U.S.A.