Configuration Management

1. Configuration Management Configuration Management (CM) is the process of controlling and documenting change to a system.

2. ‘CM is a foundation of a project. Without it, no matter how talented the staff, how large the budget, how technically superior the development tools, project discipline will collapse, and success will be left to chance.’ Source: SPMN, Little Book of Configuration Management. www.spmn.com CM as a Speed Bump

3. Content • Objective of configuration management (CM) • Why bother with Software Configuration Management ? • Functions of CM • Configuration Control Board • Role of QA • Bad excuses related to CM Adapted from: - Kasse, T., McQuaid, P., ‘Software Configuration Management for Project Leaders’, Software Quality Professional, September 2000. - Bersoff, E., “Software Configuration Management: A Tutorial.”, IEEE Computer Jan 1989, pp 6 –14. - Wiegers K., ‘Software Requirements’, Microsoft Press, 2003. - Software Program Managers Network (SPMN). www.spmn.com. - Babich, W.,Software Configuration Management. Reading, Mass, 1986, Addison-Wesley. - A.E. Fazio N.S. Shashidhara, Configuration Management forRail-Transit Operations, 21st Century Rail CorporationJersey City, NJ.

4. Sources of Changes • Requirements. • Note: The longer the delivery cycle, the more likely requirements will change (e.g. 1% per month). • Changes in funding. • Technology advancements. • Solutions to problems. • Scheduling constraints. • Customer expectations. • Unexpected opportunities for an improved system. Configuration Management Fundamentals, Crosstalk, Juillet 2005.

5. Why Bother With Software CM ? • Software is easy to change • Uncontrolled change is a common source of project chaos, schedule slippages and quality problems. • Software is invisible • Types of components to keep trace • Non executable components • Documentation • Executable components • Code • Tools used to develop and test software products • Software is often changed during the life cycle • IKIWISE (I’ll Know It When I See It) • Customer is not sure what they want • Scope creep (i.e. additional requirements) • Rework (i.e. because of errors)

6. Why Bother With SCM ? • Examples of poor practices: • The latest version of source code cannot be found. • A difficult defect that was fixed at great expense suddenly reappears. • A developed and tested feature is mysteriously missing. • A fully tested program suddenly does not work. • The wrong version of the code was tested. • There is no traceability between the software requirements, documentation, and code. • Programmers are working on the wrong version of the code. • The wrong versions of the configuration items are being baselined, delivered or installed in a product • No one knows which modules comprise the software system delivered to the customer. Babich, W., Software Configuration Management. Reading, Mass, 1986, Addison-Wesley.

7. Trends in Software Growth Boeing 777/787 Mission Control Ground Station Instructions Manned Systems 10,000 B-2 Skylab 2 B-1B Apollo 17 C 17 Shuttle F-16 c/d Unmanned Systems Apollo 7 1,000 Gemini 3 Galileo 100 Apollo 7 Viking Gemini 10 Titan Instructions = Equivalent memory locations in 1000 1 1960 65 70 75 80 85 90 95 2000 Date of Flight

8. Benefits of CM to a Project • Reduces confusion and establishes order. • Organizes the activities necessary to maintain product integrity. • Ensures correct product configurations. • Limits legal liability by providing a record of actions. • Reduces life-cycle costs. • Enables consistent conformance with requirements. • Provides a stable working environment. • Enhances compliance with standards. • Enhances status accounting. Configuration Management Fundamentals, Crosstalk, Juillet 2005.

9. Terminology: Configuration Item (CI) • CI is the most elementary entity that will be identified and managed during the lifecycle. • Élément de configuration • A set of CIs • Baseline (référentiel) • An approvedsnapshot of the system at appropriate points in the development lifecycle • Serves as the basis for further development • e.g. Entry criteria for next phase • Can be changed only through an agreed upon changeprocedure • SCI – Software Configuration Item

10. Terminology: Baselines • For Software Engineering (CMMI) • Baseline. • A set of requirements, design, source code files and the associated executable code, build files, and user documentation that have been assigned a unique identifier. • Release of a baseline constitutes retrieval of source code files (configuration items) from the configuration management system and generating the executable files. • A baseline that is delivered to a customer is typically called a “release” • A baseline for an internal use is typically called a “build.”

11. System Life Cycle Baselines Hardware System Design Software Reviews Software Reqts Analysis SRR SDR Prelim Design Detail Design Coding, CSU testing SSR PDR CSC Integration Testing Functional Baseline CDR CSCI Testing TRR Allocated Baseline Design Baseline Product Baseline Adapted from DoD Standard 2167A

12. Mapping of system and developmental baselines Kasse, T., McQuaid, P., ‘Software Configuration Management for Project Leaders’

13. Purpose of Configuration Management • To establish and maintain the integrity of work products using: • configuration identification, • configuration control, • configuration status accounting, • configuration audits.

14. Four Key Functions of CM Source: Configuration Management Fundamentals, Crosstalk, July 2005.

15. Functions of CM 1. Configuration Identification • Role: To define baseline components • Question: What is my system configuration ? • Your system configuration consists of the following items: Item1, Item2, Item3,… • Major Activities Exercice 1, 2 Critères pour sélectionner les CI ?

16. Items Under CM Control Source: Configuration Management Fundamentals, Crosstalk, July 2005.

17. Functions of CM2. Configuration Control • Role: To provide a mechanism (i.e. documentation, organizational body, procedures) for preparing, evaluating, approving or disapproving all changes throughout the lifecycle (CCB) • Example of a document: Engineering Change Proposal (ECP) • Question: How do I control changes to my configuration ? • The steps in processing changes are: Step1, Step2,…*

18. Configuration ControlBasic Ingredients • Process and Procedures for controlling changes • An organizational body • For formally evaluating and approving or disapproving a proposed change • E.g. Configuration Control Board (CCB) 3. Documentation for formally initiating and defining a proposed change • Deficiencies, hardware change, new requirement, economic savings, schedule accommodation • e.g. Engineering Change Proposal (ECP)

19. Elements of Change Control Process • Who can initiate the change requests • The individuals, group, or groups who are responsible for evaluating, accepting, and tracking the change proposals for the various baselined products • What the criteria are for placing the software components under formal change control • The “change impact” analysis expected for each requested change • How revision history should be kept • The check-in/check-out procedures • The process the software configuration control board (SCCB) follows to approve changes • How change requests will be linked to the trouble-reporting system • How change requests are tracked and resolved • The reviews and/or regression tests that must be performed to ensure that changes have not caused unintended effects on the baseline • The procedure that will be followed to update all affected software.

20. Flow of a Change Request Business Requirement Modifies drives specification of System Requirement, Use Case,External Interface Requirement, Quality attributes Business Rules Modifies Change Request is origin of is origin of Modifies Software Functional Requirement depends on another Modifies is satisfied by Leads to creation of is verified by Architecture, User Interface, Functional Design Project Plan Task SystemTest is implemented in is verified by Code Integration test is verified by Unit test Wiegers K., Software Requirements, Microsoft Press, 2003.

21. Impact Analysis Report Template. • Change Request ID:______________Title:___________________ • Description:__________________ Analyst: _________________ • Date Prepared: _______________ • Prioritization Estimates: • Relative Benefit: (1-9)___ • Relative Penalty: (1-9)___ • Relative Cost: (1-9) ___ • Relative Risk: (1-9) ___ • Calculated Priority:______ • Estimated total effort:___________ labor hours • Estimated lost effort:___________ labor hours • Estimated schedule impact:___________ days • Additional cost impact:___________ dollars • Quality impact:_______________________________ • Other requirements affected:________________________ • Other tasks affected:__________________________ • Plans to be updated:__________________________ • Integration issues:_________________________________ • Life cycle cost issues:_________________________ Wiegers K., ‘Software Requirements’, Microsoft Press, 2003.

22. Configuration Control Board (CCB) • A body that provides the means to implement change control at optimum levels of authority. • Customer’s CCB • Prime contractor’s/integrator’s CCB • Developer’s CCB • Subcontractors’ CCB • A body of people (an individual or a group) that is empowered to • Make binding decisions about which proposed changes and suggested features to incorporate into the product.

23. CCB - Membership • Project management • Development • Testing • Quality Assurance • Customer representatives • User documentation developers • Technical support • Configuration management

24. CCB - Decision Making Process • Identify: • The numberof CCB members or the key roles that constitutes a quorumfor makingdecisions. • Whether voting, consensus, unanimity, or some other mechanism is used to make decisions • Whether the CCB Chair is permitted to overrule the CCB’s collective decision • Whether a higher level CCB or someone else must ratify the decision

25. CCB - Decision Making Criteria • Financial savings or revenue • Customer satisfaction, • Competitive advantage, • Time to market. • Product cost • Delivery schedule, • Product quality • Product functionality, • Support costs, • User dissatisfaction. • Other criteria...

26. Change Request Flow with CCB • Draw a flowchart showing: • Each stage of a Change Request (CR) • The status of the CR at each stage • E.g. submitted, approved Note: The CCB is performing its functions Exercise

27. Change Request Flow with CCB Originator submits a change request Submitted CCB Evaluator performs impact analysis don’t make the change Archived Evaluated Rejected CCB decided to make the change Approved change has been made; formal verification requested verification (e.g. regression test) failed Change Made no formal verification is requested; changed product is installed change has been verified Verified modified product is installed Formal verification: e.g. inspection Wiegers K., Software Requirements, Microsoft Press, 2003 Closed Archived

28. Change Request Flow – Swimlane Notation T N PM PREPARE / T E NO IMPLEMENT CLOSE C M START SEND CUSTOMER E END / E PROJECT CHANGE END J G VARIATION ACCEPTS COMMUNICATE O CHANGE ORDER A R ORDER N P A YES M N O I T REQUEST C N / PROPOSE U F A CHANGE Y N A ADVISE PREPARE ADV. COMPLETE / NO YES CERTIFICATION CERTIFICATION END / CHANGE CLOSE CHANGE BODY APPROVES BODY COMMUNICATE INSTRUCTIONS ORDER G (IF RELEVANT) (IF RELEVANT) N I R E E CHANGE N YES Engineering I G ORDER NO N CLOSED E EXECUTE ISSUE CHANGE NO YES DESIGN CHANGE CHANGE ORDER CHANGE ORDER NOTIFICATION CLOSED CUSTOMER YES NO CCB Acme Inc. OPEN / NO CHARGE DESIGN ANALYZE A ACCEPT A END / PREPARE COST TO CHANGE CHANGE CHANGE COMMUNICATE CHANGE ORDER YES SUPPLIER Procurement YES ECR ACCEPTANCE MAY BE DELEGATED NEGOTIATE SUPPLIER IMPLEMENT END / CHANGE WITH END AGREES COMMUNICATE CHANGE SUPPLIER NO G N I Manufacturing R U T IMPLEMENT C END A CHANGE F U N A M N O I T T C C U IMPLEMENT U END D D O CHANGE O R R P T N I FAST TRACK UNDER EXCEPTIONAL CONDITIONS, THE PARALLEL ‘ FAST TRACK’ APPROACH MAY BE AUTHORIZED

29. Change Request Flow for Safety Critical Software Categorization of Changes • The basis for assigning a change category is its potentialimpact on the Global Risk Profile of Rail-Transitoperations. • Changes are categorized as follows: • Category A: A change which will definitely impact asafety criticalsystem, sub-system, or operating practice (e.g. operator training) and requires subsequent review by the Safety ReviewCommitteeifapproved by the CCB. • Category B: A change which may impact a safety criticalsystem, sub-system or operating practice, either directly orthrough an interface and therefore requires evaluation andformal recategorization by CCB as a Category A orCategory C change. • Category C: Changes which have no impact on the systemsafety profile of operations and which can therefore berecommended for implementation by the CCB. Source:A.E. Fazio N.S. Shashidhara, Configuration Management forRail-Transit Operations, 21st Century Rail CorporationJersey City, NJ.

30. Change Request Flow for Safety Critical Software Types of Changes • The deployment and operation of the Rail-Transit systemin some cases require that contractual considerations beincluded in CM process. • Changes are classified bytype as follows: • Type I. Has nocontractual implications • Type II. Has contractual implications which are technicalin nature only; has no commercial implications. • Type III. Has contractual implications which arecommercial in nature only; has no technical implications. • Type IV. Has both technical and commercialimplicationswith respect to the contract. Source:A.E. Fazio N.S. Shashidhara, Configuration Management forRail-Transit Operations, 21st Century Rail CorporationJersey City, NJ.

31. Change Request Flow for Safety Critical Software Safety review Committee Type 1 Source:A.E. Fazio N.S. Shashidhara, Configuration Management forRail-Transit Operations, 21st Century Rail CorporationJersey City, NJ.

32. Functions of CM3. Configuration Status Accounting • Question: • What changes have I made to the system ? • Your configuration and related changes at this time are: (Item1, Item2) + (Change1, Change2) + (Pending Change1, …) • What changes remain to be implemented?

33. Functions of CM 3. Configuration Status Accounting • Role • Provide a mechanism for maintaining a record of the evolution of a system (e.g. history) at any time: • Is this specification approved ? • Is this subsystem tested ? • Has a change request been approved or rejected by the SCCB? • Which version of an item implements an approved change request? • What is different about a new version of a system? • How many faults are detected each month and how many are fixed? • What is the cause of the trouble report? • Report on the traceability of all changes to the baseline throughout the software lifecycle Exercice

34. Functions of CM 4. Configuration Auditing • Question: Does the system I am building satisfy the stated needs ? • Your system, as currently built, differs from the stated needs as follow: Difference1, Difference2, …

35. Functions of CM 4. Configuration Auditing • Roles • Provide a mechanism for determining the degree to which the current state of the system mirrors the system pictured in baseline and requirements documentation • Verify that the software product is built according to the requirements, standards, or contractual agreement • Verify that all software products have been produced, correctly identified and described, and that all change requests have been resolved • Provide the mechanism for establishing a baseline • Ensure SCM process and procedures are performed Exercice

36. Functional Configuration Audits (FCA) • Objective • To provide an independent evaluation of the software products, verifying that each CI’s actualfunctionality and performance is consistent with the software requirements specification. • An FCA includes: • An audit of the formal test documentation against test data • An audit of the verification and validation reports to ensure their accuracy • A review of all approved changes (problem reporting and corrective actions) to ensure they have been correctly technically incorporated and verified • A review of the updates to previously delivered documents to ensure their accuracy and consistency • A sampling of the design review outputs to ensure that all findings were completed and incorporated • A comparison of the code with the documented software requirements to ensure that the code addresses all and only the documented requirements • A review to ensure that all testing was accomplished with appropriate test documentation and approved test data to ensure that all configuration items meet the established performance criteria

37. Physical Configuration Audit (PCA) • Objective • To provide an independent evaluation of the system CIs to confirm that each item that makes up the “as built” system maps to its specifications. • To verify that the software and its documentation are internally consistent and ready for delivery to the customer or end user. • A PCA includes: • An audit of the system specification for completeness • An audit of the FCA report for discrepancies and actions taken • A comparison of the architectural design with the detailed design components for consistency • A review of the module listing for compliance with approved coding standards • An audit of the manuals for format completeness and conformance to systems and functional descriptions. Such manuals include user’s manuals, programmer’s manuals, and operator’s manuals

38. Traceability Audits • To verify throughout the software development process: • Can the software requirements be traced back to the system requirements allocated to software? • Can the architectural design be traced back to the software requirements? • Can the detailed design(s) be traced back to the architectural requirements? • Can the code modules be traced back to detailed design modules? • Can the module tests, integration tests, and system tests be traced back to the software requirements?

39. FCA and PCA Source: Kasse, T., McQuaid, P., ‘Software Configuration Management for Project Leaders’, Software Quality Professional, September 2000.

40. Software Library System • Supports multiple control levels of SCM • Managers, developers, SCM, QA, Systems engineering, etc. • Provides for the storage and retrieval of configuration items • Provides for the sharing and transfer of configuration items among affected groups and among control levels within the library • Provides for the storage and recovery of archive versions of configuration items • Provides service functions • checking status, verifying the presence of all built items, and integrating changes into a new baseline • Ensures the correct creation of products from the software baseline library • Provides for the storage, update, and retrieval of SCM records • Supports the productionof SCM reports • Supports the tracing of requirements, forward and backward, throughout the life cycle • Provides safe and secure storage for configuration items (key project components) so they cannot be changed without authorization.

41. Processes needed for CM by CMMI Jonassen Hass, A., Finding CM in CMMI, Crosstalk, Juillet 2005.

42. Subcontractor Control • Ensures that the subcontractor is able to maintain the integrity of the subsystem it has contracted for, including: • Placing necessary life-cycle products under configuration control to ensure consistency with the main development effort • Maintaining a subcontractorsoftware library that will release the agreed upon configuration items or subsystems to the contracting organization

43. 16 14 12 10 8 6 4 2 0 0 5 10 15 20 Sample Chart of Requirements Change Activity Number of Requirements Changes Weeks After SRS was Baselined Wiegers K., Software Requirements, Microsoft Press, 2003

44. Marketing Management Customer Software Eng. Hardware Eng. 5 Testing 0 Sample chart of requirements change Origins Number of Requirements Changes 30 30 25 25 20 20 15 15 10 10 5 0 Change Origin Wiegers K., Software Requirements, Microsoft Press, 2003

45. Evolution of Requirements Status • Coverage ratio of higher level requirements • Coverage ratio of requirements by associated requirement tests

46. Change Control Policy -1 • Elements ofa change control policy: • All requirement changes must follow the process. • If a change request is not documented according to this procedure, it will not be acted upon. • The change control board (CCB) designated for each project will decidewhich changes to implement. • Simply requesting a change does not guarantee that it will be made. • The contents of the change database shall be visibleto all project stakeholders.

47. Change Control Policy -2 • Elements ofa change control policy: 4. The original text of a change request shall notbe modified or deleted from the database. 5. Every incorporated requirement change shall be traceable back to an approved change request. 6. Nodesign or implementation work will be performed on unapproved requirement changes other than feasibilityexploration to assist with making the approval decision.

48. Typical SCM Plan • The scope of the plan, including the project, the software to be developed, and the life-cycle phases • The relationship between the SCM plan and the other standards or plans that describe how the project will be managed • e.g. software development, SQA plan • SCM roles and responsibilities • Configuration identification • Baselining • Configuration control • Configuration management status accounting • Interface control • Subcontractor control • Software configuration audits • Software library IEEE Standard 828 – 2005 Configuration Management Plans

49. Real-World Considerations • Management Commitment • Essential to achieving benefits of CM • CM Staffing • Enough experienced people • Tooling • See ISO CM Tool Requirements Std • Establishment of a CCB • A starting point in instituting change control • Avoiding the Paperwork Nightmare • Stakeholders should agree on the amount of paperwork needed to achieve mutual level of visibility and traceability

50. Role of QA • Audit the software development activities and product • i.e., Functional Configuration Audits (FCA) and Physical Configuration Audits (PCA) • Certify CM compliance with Project plan, CMPlan, standard, processes and procedures. • Participate in the CCB and Interface Control Working Group