Software Configuration Management A Road Map

1. Software Configuration Management A Road Map Jacky EstublierGrenoble University - France

2. In the Small In the Large In the Many In the Wide Programming Languages Process Support Deployment, Variation Concurrent Eng. Distributed Eng. Composition Building Evolution Web Remote dev Activities Software Product Software Engineering, Programming, and SCM ConfigurationManagement

3. What is CM CM is the CONTROL of the EVOLUTION of COMPLEX systems Any complex system has : • Large number of components and their versions • Large number of people involved (up to 1000) • Very hard time constraints (cycle down to 4 month) • Strong concurrency (many simultaneous changes) • Strong quality constraints • Long product life duration (10 to 20 years) Goal: Keep evolving product under control and help satisfy delays and quality constraints

4. Configuration Management Managing a repository of components • Version Control • Product Models • Composition and selection Help engineers in their usual activities • Building / rebuilding (derived object control) • Work Space control • Tool control and automation Process control and support • Change control • Cooperative work • Process support

5. Branch Variant 3.2.1 3.1 3.2 3.3 Successor Révision 1 2 3 4 5 Component repository:Version control early 70s • History management • Delta (space saving) • Multi-user “control” • Merging • Still the base of almost all SCM systems.

6. Current state of practice • History: A comment + date + user name on Check-in limitations: • Only on CI/CO (hardwired event) • Only a comment (hardwired content) • Difficult to retrieve / interpret info • Trends: • More event based and DB supported. • Delta: Saving space by storing only differences between revisions. • Limitations: • Current algorithm works only for text file (line based) • Trends: • New algorithms better performance and binary deltas.

7. Multi-user control : A lock is set on check-out revision. Limitations: • It prohibits concurrent engineering!! • No way to define how teams synchronize!. Trend : • Concurent work must be allowed. • More sophisticated ways to control (no locking)

8. File Merges Based on a line to line difference + common ancestor. Limitations: • Text file merge only. • No semantics. The result is a set of lines!!. • The result can make no sense (Merging a NT with a Unix source file) Trend : • Attempts to use more semantic in merging objects=> too slow and language dependent.=>No change expected in the near future. • Complex object merges expected in the near future

9. Version Set • A group of changes has a logical name (BugFix21) • A specific release R2 can be built from a previous one R1 • R2 = R1 + BugFix21 + Extention2 – Extention3 • We can build versions that never existed before ! • Higher conceptual level, easier to understand … • For long presented as the opposite to the version tree • Is becoming available in new tools, with a version tree.

10. Version Management State of practice: A revision tree. No version concept but a mechanism. Each application use it to manage its own versioning. Question: Why are 2 objects versions of each other • They have something in common (but what?) • They are different (in what?). With RCS the answer is: • They share a large number of lines, • They have some lines which are different. Only a good answer to these questions may allow better version management.

11. Other versioning approaches (1/2) The DB Approach: Damokles (87-90): An object contains attributes. All versions together constitute a logical object. Some attributes are common to all versions (what is common) Some attributes are version specific (what is different) The Adele model. The DB approach+ three dimensional. Part of the data Model : Unified versioning and merging. Logical: Different variants. Historic: Evolution of an object. Cooperative: Concurrent activities.

12. Versioning approaches Trends : • Object versioning (unified versioning). • Hide low level mechanism (revisions and branches) • Show more semantic concept of version Issue : A good versioning requires versionned DBs ? Versioning in the Data Model or not?

13. Component repository: Product models • A good CM is possible only if the system knows about the • objects managed. Current state : • Data Model : files and directories • We need a good DB one (incl. Relationships). • Product Model : System Model (80s) : • architecture / composition of a configuration : • Relationship between components, dependencies, • compatibility constraints, … (MIL ADL). • Most often only a Makefile contains that information!!.

14. Architecture Model (ADL?) Component n y o t i i M t c c i System Model C Variant e r g l S e n e n i S Bound Model d e Revision r g n e But still Conditional Compi. i l f i B o p f e Compiling Binary linkable o m e r r e o e g k e C e r Linking n Binary locatable r i g D e e L d D a Loading Memory Image o L Dyn. Lib. Executable Image Help engineers: Selection/Building

15. ADL SCM Conf. CDS Conf. Software Management Distr. Syst. Architecture Composition °°° ° °° Behaviour ° Consistency °° ° ° Building ° °°° ° Versioning °°° ° Selection °° Convergence, intégration, complement, interoperability ??.

17. Architecture vs SCM • Are ADLs just other « programming » languages? • And managed by SCM as any other one! • Should IDL be part of the SCM language ? • Product model always embodies part (all?) the product architecture • Is it possible/desirable to put all features in a single lang.? • If not : • What are the orthogonal dimentions (versioning, dynamic, ) • Should the split be based on • Logical dimentions • Physical dimention • User roles (developper vs release mgr vs ..)

18. Work Spaces NT view Unix View Help engineers: Work Space Control Component Repository DB view Problem: Define and control each arrow: Concurrent Engineering control

19. Explicit Mapping by Copies A WS manager controls the copies back and forth between WS and DB. RS CM tool WS WS manager Object set (configuration) Global CI/CO Most commercial CM system are in that class Pros: No overhead in WS; WSs can be managed simultaneously Cons: The WS model is low level; Collaboration is ill supported.

20. WS = (virtual) FS = RS If the WS is in the Repository, there is no need for a mapping • Adapting the File System (virtual file system) • Drivers (Clear Case), I/O Libraries (3D, nDFS), NFS. • Pros: • Implicit and transparent version selection. • Easy adaptation for users. • Cons: • The FS must be changed • A single RS/WS mapping is possible. • Encapsulating all tools

21. Current Limitations (1/2) A central data base as repository. => Severe scalability and efficiency limitations. Too low level product models Single rigid projection. Concurrent work definition and control => No explicit definition, No high level concepts =>No way to enforce policies =>Strong limitations in distributed work

22. Current Limitations (2/2) • Need for : • Really decentralized solution (no central DB) • Multiple structure, format and organization. • Simpler, independent and more efficient WS managers • Really high level product model • High level control of highly concurrent / distributed work.

23. Process Control and Support • Belief:Good knowledge and control of the software process is mandatory for quality, cost and delay predictability. • SCM Processes : • Change Control. Well identified and formalized, => Each SCM tool proposes a solution. • Concurrent engineering Team coordination and synchronization. => No satisfactory proposition. • Others : Business, administration, organization => Weak and overlaps with other tools.

24. State of Practice in SCM A process is described by a “State Transition Diagram” (STD) of the product. Resolved Concluded Entered InReview Assigned Forwarded Defered Rejected It is a fine grained approach, but too low level.

25. Product Oriented view of Process Almost all SCM product support that style. Most often the STD is hardwired into the system Sometimes it is user definable. The STD is a “product oriented” view of a process (the product evolution is controlled). The STD is a sparse view of the process. No global view. It is a very limited view of process. It is a fine grained approach, but too low level.

26. Activity oriented view Data flow and control flow modeling. User and resource allocation.

27. Activity oriented (2/3) Global view. Better for management, not for product control. A few products offer also an activity oriented view (continuus, Clear case ...), but the integration of both views is often week. A process should be described at a high abstraction level, it mustdefine many other aspects.

28. Activity oriented (3/3) Idealy: data flow, control flow, STD, workflow, cooperation, WSs, agendas, product models, agents, teams, roles, measures,.... Furthermore, it must be executable, measurable, tunable, reusable, evolvable.... We are far away... but it is the only domain, in SE, where process support proved to be successful.

29. Activity oriented view Data flow and control flow modeling. User and resource allocation.

30. Product Model

31. CD and DF

32. Object and activities: Local STD

33. WS cooperation policies

34. Measures (instances level)

35. Evaluation and enhancement

36. Computer Aided Engineering Spécific CA Env. & tools CAD Soft. Eng CAM Product ModelEvolution ModelCooperation ModelProcess Model Computer Aided Engineering Platform Middleware, BD … OS

37. Product Model Bicycle Frame Front Wheel Rear Wheel tyre Spoke Spoke Spoke Backbone : the composition relship. Relphip handling is fundamental.

38. The front wheel Bicycle Bicycle Bicycle The last spoke of The front wheel 1 2 Frame Wheels Frame Wheels Frame Wheels 32 The last spoke 1 Tyre Tyre Spoke Tyre Spoke Spoke Quantified BOM Relative occurences BOM Different levels of composition

39. PDM Versioning : logical Historical as in SCM Light Bicycle Trial : &Qty = -2 Dynamo Qty = 2 Frame Mudguard Rear Wheel Front Wheel Seat Seat Post Gear Wheel Rapid Fix Wheel Carbon wheel Composition Alternate Substitute

40. PDM Versionning : effectivity • Effectivity by • Date • Date range • Version

41. PDM Versionning : effectivity • Other possible effectivity strategy • Destination vs Origin effectivity ({1,3},{2,5}; …. • Revision vs variant effectivity Trail, {Carbon, HardSteel} • Local vs global effectivity • Complete vs partial • PDM : Origin, Revision, local, complete Object X ({1,3}, 5), 1; 4,2; {5,*}, 5 Object Y

42. PDM Versioning : domains Light Bicycle Dynamo Frame Front Wheel Rear Wheel Seat assembly Wheel Seat Seat Post View : As-designed View : As-planned View : As-designed-electricaly View : As-designed + As-planned View : as-designed + as-designed-electricaly

43. PDM Selection • Selection is a filter applied to a variation dimention • A context is the intersection of filters applied on all dimentions • view (selecting the views), • effectivity (selecting the revisions), • option (selecting the options), • alternate and substitute (selecting the variants) and a • life cycle. • A context is dynamic : a DB view! • Week cooperative control (no work space)

44. PDM vs SCM : Summary • Fundament differences : • Model vs Product. A software is both a model and the product. • Structure. Constrained versus Unconstrained. • Accidental differences: • SE is evolving very fast. Innovation and responsiveness are priorities • PDM is more mature. Standards and stability are priorities. • Short term priority for both : • Usability (too expensive to tailor, too difficult to use). • Efficiency (too slow for large products and teams). • =>Integration will be done only if imposed by customers.

45. Global assessment • Strength : A successful technology because .. • It does not take into account the syntax and semantics of programming languages, applications, … • => Simple, generic and efficient tools. • => Heuristics proved to be very effective. SCM is a pragmatic approach to solve SE problems. • Weakness • It does not take into account the syntax and semantics of programming languages, applications, … • => No correctness criteria. • => Functionalities are limited.

46. Challenges 1 : Interoperability • A growing number of tools include : • a data repository, versioning, concurrency control Example : most 4GL tools, PDM tools, editing tools • process support Example : MS Project, Lotus, workflow, … And the increasing number of COTS tools, the increasing need to make interoperate tools /activities. • SCM tools must be designed as a component which interacts • with many others, both for data AND process control reasons.

47. Challenge 2 : SE Practices • Buying COTS becomes common • evolution, availability, development process etc. are not controlled • Component based SE : custom components • Construction by assembling COTS and custom components • Non source code ways of adapting / connecting components • SE evolution : From • centralised control over owned pieces of source code to • decentralised control over foreign components SCM systems does not provide services for that !

48. Challenge 3 : PDM v.s. SCM • The increased part of software in any industrial product • The need to develop consistently software and hardware • Increasing overlap / conflict with PDM tools • Many similarities but • Many differences … • None can pretend to do both well. • There is a need to reconcile these worlds. How ? • Integration • Cooperation development process development process • Extension …

49. Challenges 4 : Remote development The gap between local, distributed and remote development should (almost) disappear. • The virtual enterprise (distributed and heterogeneous) rises now problems. • Issues : Efficiency, scalability, remote concurrent engineering control, heterogeneity, …

50. Challenges 5 : New domains New applications : ex Managing web pages • Potentially a larger market than Soft development • New / different issues • more pages (x10, x100) • faster changes (daily) • more (undisciplined) contributors (x100) • contributors are not soft. Professionals Who will take that market : Document management tools, SCM tools, others?.