Best Practices on Incorporating Quality Assurance Into Your Software Development Life Cycle

1. Best Practices on Incorporating Quality Assurance Into Your Software Development Life Cycle Katya Sadovsky, Applications Architect Carmen Roode, Associate Director of Systems Development Marina Arseniev, Associate Director of Enterprise Architecture University of California, Irvine

2. University of California, Irvine • Located in Southern California • Year Founded:  1965 • Enrollment: over 24K students • 1,400 Faculty (Academic Senate) • 8,300 Staff • 6,000 degrees awarded annually • Carnegie Classification:  Doctoral/Research – Extensive • Extramural Funding - 311M in 2005-2006 • Undergoing significant enrollment growth

3. Today’s World What’s key with today’s systems • Security – data and reputation • Expectations are different – accessibility, availability • Systems tied to employee mobility • 24 hours a day • No training, intuitive, flexible/changeable • SDLC needs to support more complex environment – distributed computing environment, different architecture, components, web services • SDLC has to reflect today’s business environment

4. What we still see Examples of avoidable problems • Systems failing during peak usage • Insecure systems – break-ins • e-Voting debacles

5. Quality Assurance within the SDLC Common life cycle phases • Budget & Staffing • Requirements/Prototyping • Design • Development • Testing • Roll-out • Maintenance • Decommissioning of the system

6. Agenda • Education and Staffing • How-tos in the Software Development Life Cycle Process • Templates and Checklists • Outcome/Summary • Useful URLs and Q&A

7. Education and Staffing • Train staff on SDLC • Train on minimum competencies • Technologies – i.e. Java, databases • Project management • UML • Certifications • Ongoing training budget allocation for all staff

8. Education and Staffing • Split out project by experience level • Coding of key software done by more experienced staff (i.e. authentication & authorization) • Code and design reviews • Training and mentoring • Have minimum technical competency hiring standards

9. Business Unit Education • Educate business units on acquiring secure and quality systems • Business modeling and functional requirements • Knowing security issues, laws, regulations and avoiding bad practices: • Regulated or confidential information can be unintentionally transmitted through email • Private data can be entered into a text field

10. Agenda • Education and Staffing • How-tos in the Software Development Life Cycle Process • Templates and Checklists • Outcome/Summary • Useful URLs and Q&A

11. Software Development Life Cycle Without QA

12. The Software Development Life Cycle • Traditional waterfall method puts QA at the end of the development process • Performance testing after QA • What you find • The system functions correctly but it’s too slow to use • This means architecture solution is incorrect • Example – object to relational re-architecture required for performance

13. The Software Development Life Cycle How to embed QA into your SDLC • Documented formal steps • Templates • Checklists • Reviews • Approvals • Formal Change Control

15. Integrating Quality Assurance in Requirements

17. Integrating QA in Requirements Acquisition or development • Users to identify needs, document requirements • Prototype what they want • Identify things that can change • Identify business service constraints • Acceptable levels of downtime • Acceptable levels of data loss • Business cycles • Identify Security requirements (examples: Compliance requirements – PCI, SB1386, FERPA, HIPAA, Risk assessment – normal or high risk application?)

18. Integrating QA in Requirements • Know your business cycles • Students – know cutoff dates for students • Financial – know fiscal cycles, month end, year end • Payroll – hiring waves related to academic year, paycheck generation & view • Plan on peak usage during load testing

19. Integrating Quality Assurance in Design

20. Integrating QA in Design • Costliest mistakes to fix are introduced in design • To ensure QA, SDLC must include a design review process • Use your most experienced staff! • Checklist for adherence to architecture and technology standards • ex: Java, .Net, Portal, new technology? Exception? • Check for appropriate design solution – use rubric  • If 24/7 is required, what needs to be in the design to assure Quality? • scalability, availability, accessibility, redundancy? • design for clustering, fault tolerance, load balancing…

21. Integrating QA in Design – Design for Reuse • SDLC process must promote identification, design, and reuse of common tested components to achieve QA • Requirements phase extracts reuse candidates for design • document management, user/group management, workflow… • Design phase must identify and provide solutions for vulnerable points • authentication and authorization/access control • database or file stores of sensitive data • logging/auditing • QA has to test the ability to do forensics • Design security solutions with forensics QA in mind • QA your encryption, auditing, and logging design

22. Integrating QA in Design – Design for Testability • Design system and code for load testing QA • Load testing must test redundancy, clustering and peak concurrency requirements • code is written differently for load testing • Determine how you will test and write test plans at design time! • Distributed components / Web Services testing • QA is done at different layers – network, database, messaging • Design for Disaster Recovery QA • SDLC Design phase must guarantee Testability

23. Integrating Quality Assurance During Implementation, Testing, and Remaining Phases

24. Integrating QA During Implementation Implementation – make QA “routine” • Schedule code reviews • Require developers to build unit test harnesses – Junit • Automate nightly code and application security scanning – Jtest, AppScan, Nessus, database security scanning • Schedule network and configuration scanning - Foundstone • Write and use manual security test procedures • Perform concurrency and stress testing - Jmeter, OpenSTA • Integration testing • Services and APIs • Are services or distributed components using encryption? • How does an application authenticate to a service?

25. Integrating QA During Testing • Functional testing • Do you use formal Test Plans or AdHoc? Tied to Requirements? • Done by developers and end users? • Do Pilot Users test methodically using Test Plans? • How do you ensure testing coverage is adequate? • SQL Injection and Cross-site Script testing • Browser Compatibility Testing (ex: browser cache) • Regression testing

26. Integrating QA in Remaining Phases • Deployment • Helpdesk, Sys Admin, support staff cross-trained? • Policy issues identified? • System and data backups, disaster recovery • Operations/Maintenance • Repeated “routine” reviews and scanning • Change control • Decommissioning of Application and Data • Retention/preservation of information and data • Sanitize media, properly dispose hardware and software

27. Agenda • Education and Staffing • How tos in the Software Development Life Cycle Process • Templates and Checklists • Outcome/Summary • Useful URLs and Q&A

29. Templates and Checklists

30. What certification or audits does the University have that the system will be managed per our guidelines and contract agreement? How do you manage the system for detection of intrusion. How often is the system patched, by whom and when? How are we notified if system security is breached? Notification handling? How is data purged from the vendor's hardware? How are disks, tapes, or computers that might store sensitive data disposed of? Are the media erased before disposal or reuse? Where is the hardware location? Is it inside or outside of the United States? Is it subject to our laws? Are the personnel who administer and use the hardware located within the United States and subject to our laws? Is data encrypted? If private data is transmitted, either via Internet, on CD-ROM or file transfer, is it encrypted? Is SSL enabled to the application so that traffic over the Internet, including authentication is secure and private? Data loss, data backups: what are the guarantees? Are backups stored offsite? If backups have sensitive data, are the backups encrypted? Can we store the backup at UCI? How about disaster recovery planning? How is the hardware or database distributed by the vendor among customers? Is one hardware used for all customers? Is a single database used for all customers or does each customer have a private database? How are user accounts managed? ASP Vendor Security Checklist

33. Our Change Control Process • Coordinate and schedule changes in network, database, applications, OS, firewalls and configurations • Avoid downtime due to collisions • Avoid accidental security exposures • We use Oracle Calendar • All developers, system and network admins meet every Tuesday morning for at least 15 minutes! • 2 week notice of all planned changes • Test Plan and checklist required • Identification of required security tasks • High/low risk identified on all changes • Changes recorded in AdCom Service Desk

34. Agenda • Education and Staffing • How-tos in the Software Development Life Cycle Process • Templates and Checklists • Outcome/Summary • Useful URLs and Q&A

35. Achieved Outcome

36. Q&A Useful Links • AdCom’s SDLC Guidelines: http://snap.uci.edu/viewXmlFile.jsp?resourceID=1535 • Development for technology professionals: http://snap.uci.edu/viewXmlFile.jsp?resourceID=1433