Dean Knudson Alan Braaten 3/26/2009

1. Industry/University Cooperation in Defining Software Processes for use in Real-world Computer Science Capstone Team ProjectsSEPG – 2009 Dean Knudson Alan Braaten 3/26/2009

2. Outline • Capstone Projects at North Dakota State University • University - Industry Cooperation • Process Model for CS Capstone Class • Current Status of Processes at NDSU • Next Steps

3. Background – CS Capstone Projects • North Dakota State University • Fargo, ND – over 13,000 students • BS, MS, PhD (CS and SW Engineering) • 200 CS undergraduates, 180 graduate students (~60 in PhD program) • Capstone Projects • CS Majors – final semester • Done in teams of 3-4 students • Over 60 projects done for regional companies in the past 5 years • Learn best by doing

4. Background - CS Capstone Projects • Sponsoring companies include the following: • IBM, ATK, Thomson Reuters, Cargill, Honeywell, Polaris, Phoenix International, Sundog, Forum Communications, Rockwell Collins, West Corp., Noridian, CNSE, NISC, Microsoft • In the fall projects are defined with the sponsoring companies • Have real value to the customer • Not on a critical path • Do not require too much domain knowledge • Appropriately sized for a small team working for one semester

5. Sharing Proprietary Information Among Instructor/Team/3rd Party

6. Background - CS Capstone Projects • Follow semi-formal Project Management and Software Development processes • For software development - follow CMMI Level 2 guidelines • We follow the tenant that the success of a software system is highly influenced by the quality of the process used to develop and maintain it • For project management use the five standard phases • Set multiple mini-milestones • Provide many templates and examples to help the students

7. Background - Sample Projects • Simulate a Radio Interface • It included a user interface that allowed users to construct and send messages as well as log and display all message traffic. • The system included support for error checking (ACK/NAK, CRC) of messages. • The interface could be configured for baud rate, com port, parity, # of bits, # stop bits, etc. radio RS-232 munitions

8. Background - Sample Projects • Election Results Visualization • System for use on the website of a news/communication company. • Displays election • Displayed on a precinct level map in a color coded manner. • Color shading was used to indicate results • Actual vote counts are also shown for each precinct as that precinct is scrolled over.

9. Outline Capstone Projects at North Dakota State University University - Industry Cooperation Process Model for CS Capstone Class Current Status of Processes at NDSU Next Steps

10. University – Industry CooperationAt NDSU • The first four years of developing our software processes were mainly done internally • The instructor had over 35 years of industry experience and including working with SEI level 3 organizations and managing SEPG teams • Last year we obtained a small grant that allowed us to do a gap analysis of our capstone processes • This showed many areas where we would not pass an assessment (many because our process was not sufficiently documented)

11. University – Industry Cooperation • This year we started working with several sponsoring companies to evaluate our processes/templates • Our goal is to have a well defined set of processes that • are appropriate for CS capstone projects • would meet SEI level 2 standards • are close to what our industry sponsors use internally Capstone Projects New processes and templates Companies NDSU Evaluate Feedback on processes and templates Feedback from projects

12. University – Industry Cooperation • Background needed to allow for good cooperation • Create a first version of a software development process and use it on capstone projects • Establish a record of producing good results in capstone projects following these processes • Determine where there are gaps on your processes • Invite interested sponsors to take part in a review of your processes and to help define how to fill the gaps in your process • Make sure everyone has the ability to be heard and that you seriously consider every recommendation

13. Outline Capstone Projects at North Dakota State University University - Industry Cooperation Process Model for CS Capstone Class Current Status of Processes at NDSU Next Steps

14. NDSU Capstone Process Overview Initiation • Five Project Management Phases Planning Execution Req. Control Design Code Test Closure

15. Project Initiation Phase • Project Initiation Document - Negotiated with the sponsor at the outset of a project to identify the scope and nature of work to be performed. • Make sure the team understands the purpose of the project • Understand big picture, not just what is to be done but why does the sponsoring company want this project • Identify the goals and objectives of the project • Identify project risks, assumptions, dependencies and constraints • Define basic project setup decisions • E.g. Sponsor requires code to be done using C#, non-disclosure statements need to be signed • Identify the high level requirements/features and negotiate status with sponsor • Students are not used to the concept of being able to negotiate their work assignments(commit, target, no-commit) • Define the project deliverables • Identify resources needed • This document is essentially a Statement of Work (SOW) for the project

16. Project Planning Phase • Project Plan Document - Identify the staffing, communications, tracking and computer resources needed • The staffing plan is the main component, MS Project is used by all teams and the plans are updated throughout the semester • A design methodology is implied in the plan since it will vary depending on whether waterfall, spiral, agile, etc. is chosen • More detail is included in near term tasks (detail in later tasks will be added as the semester progresses) but overall expectations are given • The communications plan identifies all planned meetings (e.g. weekly conference call with sponsor), defines who needs to share information and how it will be done (e.g. proprietary website for the team) • The tracking plan identifies standard metrics to be collected (tracking size/effort is a new activity this semester) • Computer resources needed defines any special lab/hardware/software needs

17. Project Execution Phase • This is where the real work of the project is done (Req./Design/Code/Test) • Open to any design methodology (waterfall, spiral, agile, etc.) • Based on the Statement of Work • There are several tools and templates associated with this phase (Req./Design/Test Plan documents, Coding standard)

18. Project Control Phase • Done in parallel to Project Execution • “The purpose of Project Management and Control is to provide an understanding of the project’s progress so that the appropriate corrective actions can be taken when the project’s performance deviates significantly from the plan.” (from CMMI Second Edition) • We try to keep this to a minimum but include enough information to understand the project’s progress • Weekly status reports • Updated project plans showing effort spent and % complete on each task • Bug/issue/risk tracking (system provided to do this) • Weekly meetings with sponsor or mentor • Monthly one-on-one team meetings with instructor • (Can’t quite do management by walking around) • Periodic meetings with process auditor • Mid-term and final formal presentations • We are adding size and effort tracking this year

19. Project Closure Phase • This phase is simpler in capstone projects than in the “real-world” since students must finish their projects by the end of the semester and automatically move on to new challenges • For capstone projects, the main issue is to make sure all deliverables have been successfully passed on to the sponsor • Deliverables include • Requirements and design documents • Code • Tests and test results • and possibly • Regression tests • Build processes • Install processes • User’s manuals and on-line help files • All projects perform a post mortem and the results are also passed on to the sponsor (new this year is reflection on tracking size and effort) • The sponsor/mentor must sign off on this document

20. NDSU Capstone Process Overview • Process Feedback From Industry • Define Shall/Should/Will/May/etc. for use in Requirements Documents • Have a default coding standard (many suggestions and feedback on draft documents) • Size and Effort must be tracked (several detailed comments on this) • Need for a CCB and how to manage a CCB (only for major changes, team/sponsor/instructor) • How to deal with the concept of a baseline (set at delivery to customer) New processes and templates Companies NDSU Evaluate Feedback on processes and templates Feedback from projects

21. NDSU Capstone Process Overview • Templates are used extensively (along with examples from previous years) • Project Initiation • Project Planning • Project Schedule • Software Requirements • Software Design • Bug-Action Item Tracking • Software Test Plan • Weekly Report • Mid-term and Final Reviews • Project Closure • Postmortem

22. NDSU Capstone Process Overview • Tools/Support • Used Trac for company/team proprietary information and document control • Used Trac for bug/issue tracking • Used Trac (SVN)for configuration management of code • Blackboard for process documents, class lecture material • XXX for checking code complexity • YYY for code counting

23. Outline Capstone Projects at North Dakota State University University - Industry Cooperation Process Model for CS Capstone Class Current Status of Processes at NDSU Next Steps

24. NDSU Capstone Process Current Status • Problem Area - Effort estimation using size • Some but not all students are exposed to PSP before taking this class • Track effort/size planned vs actual on project (don’t evaluate individuals by LOC produced or hours spent) • Problem Area – Organization level metrics • Need to define appropriate metrics to be kept at the department level • Evaluations from companies/students • Size/Effort data

25. NDSU Capstone Process Lessons Learned • Sponsors need to really care about the processes used as well as the delivered software • Use of real-world projects is invaluable – real customers will be taking over the code and maintaining it • Providing templates and examples is very important • Providing tools for document and code management as well as bug tracking is important

26. NDSU Capstone Process Lessons Learned • Experience gained by students from following good software engineering processes is invaluable • Capstone classes are a lot of work to run well but are worth the effort • Set high expectations and expect good results (instructor and sponsor/mentor)

27. Next Steps • Get feedback from sponsors/mentors and incorporate good ideas into our processes • Monitor size/effort estimation processes and refine them as necessary • Perform formal process audits of projects throughout the whole life-cycle • Establish organization level metrics • Run capstone projects with the full process and collect data that can be used in an assessment • Perform another gap analysis then possibly an informal level 2 assessment • Perform a formal level 2 assessment – if passed, move on to the next level

28. Conclusion • It is possible for universities and industry to cooperate to define software development processes to be used on computer science capstone projects • Not only does this result in a better relationship between the two but by using the resulting processes students are better prepared to enter the workforce