1 / 29

What does Quality Assurance Mean to Sakai?

What does Quality Assurance Mean to Sakai?. Alan Berg Megan May Seth Theriault. Topics. What does a QA Director do? QA Today Who is in the WG Types of testing currently done and release activities Strides forward Resources available QA Network 2.5 Release Activities Update

illiana-bentley
Download Presentation

What does Quality Assurance Mean to Sakai?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault

  2. Topics • What does a QA Director do? • QA Today • Who is in the WG • Types of testing currently done and release activities • Strides forward • Resources available • QA Network • 2.5 Release Activities Update • QA Tomorrow • Questions

  3. Role of QA Director • Coordination! • Help bring parties interested in the same testing efforts together • Foster initiatives that support quality throughout the community • Assist in bridging the gap between test and design & development teams • Ensure release activities are transparent

  4. QA Today • Focus on Functional Testing

  5. Who is the QA WG? 2.3.0 44 People 17 Institutions 5 Countries 2.2.1 23 People 10 Institutions 4 Countries . . . How do they get involved? 2.4.0 93 People 37 Institutions 11 Countries 2.5.0 56 People 17 Institutions 4 Countries

  6. Strides Forward • Deployment testing on wider variety of platforms • Integration of Accessibility WG activities in general QA cycle • Expanding/updating library of test scripts • Reflection on Release process • http://confluence.sakaiproject.org/confluence/x/fbU

  7. QA Network Amsterdam Boston U Cambridge Cape Town Columbia Indiana rSmart Georgia Tech Coming soon servers in Japan and Australia

  8. CollabHow to be in the know! https://collab.sakaiproject.org/portal

  9. Jira http://jira.sakaiproject.org/

  10. Confluence http://confluence.sakaiproject.org/confluence/display/QA/Home

  11. Perceived Quality Level

  12. Quality, like beauty, is clearly in the eyes of the beholder.

  13. Perceived Quality Poll

  14. QA Tomorrow • Sakai Community Concerns • Several prominent campuses experience serious performance problems with 2.4 • 2.5 Release Update • Further refinement of process • Goals for 2.5

  15. Paths Forward • Common set of testing vocabulary • Unit Testing • Integration testing • System Testing • Performance • Load testing (lmp@umich.edu) • Integration testing http://confluence.sakaiproject.org/confluence/x/M4B5

  16. Roadmap for Tomorrow

  17. Metrics to Describe Quality 2.2.0 – 85% Verification (504 JIRA Tickets) 2.3.0 – 85% Verification (611 JIRA Tickets) 2.4.0 – 82% Verification (1,345 JIRA Tickets) 2.5.0 – Past bug verification

  18. Metrics to Describe Quality

  19. Static code Review • Search for bug patterns • What to process: Source code, Binary • Two types of bug pattern • Generic • Classic (buffer overflow, input validation) • Contextual • Language specific • Framework specific • Configuration specific • Proximity effect

  20. Hinting nightly http://qa1-nl.sakaiproject.org

  21. Random Example

  22. Duplicate Code

  23. Do we wish to use the reports more aggressively? Are the developers taking note? Bugs reported are not always correct Bugs found not always important But duplicate code, unit test coverage, and failing to deal properly with exceptions hint strongly at quality. Not enough Contextual searching for Sakai and spring configuration specific issues Reports do not explain enough Answer probably not, but thankfully trends in the marketplace will improve the quality of results overtime.

  24. A year on • Update prototype • Less noise more explanation • Graphical tracking of change • Recommend Eclipse Plugins • Look at emerging products • See as part of a series of QA processes. • Agree on metrics to act on and then act.

  25. A good book to read • Secure Programming with Static Analysis • Brian Chess, Jacob West • Addison Wesley • ISBN: 9780321424778 • Includes CD with software and examples.

  26. What you can do • Pledge Resources • What happens in the community has a direct local impact • Easy point of entry into Sakai • All skill sets are needed • Discussion • Why hasn’t this been a priority? • What are obstacles in the way and how do we remove them?

  27. Questions

  28. Other Sessions of Interest • 2.5 QA Workshop (Thursday, 13:00 - 14:30 in Salon 5) • The Automated Testing Conundrum in Sakai (Thursday, 15:10 - 16:10 in Salon 5)

More Related