Component-Based Software and Commercial-Off-The-Shelf

1. Component-Based Software and Commercial-Off-The-Shelf Angelo Susi Fondazione Bruno Kessler - IRST

2. Motivations One of the more significant changes in software development practice over the past years is the greatly increased emphasis being placed on building systems incorporating pre-existing software in order to keep overall development and maintenance costs as low as possible, decrease the development effort and avoid to reinvent the wheel Some drawbacks • In general source code is not available • The evolution of the code is not under the control of the application developer (dependencies arise) • …..

3. Information Sources Software Engineering Institute (SEI) - Carnegie Mellon University (CMU)http://www.sei.cmu.edu Center for Systems and Software Engineering (CSSE) - University of Southern California (USC)http://csse.usc.edu/csse/ Federal Aviation Administration Software Engineering Resource Center (FAA-SERC)

4. Agenda Component-Based Systems Impact on the software development process Commercial-Off-The-Shelf (COTS) systems/components Five Panel Model for COTS selection and adoption Cost models for systems/components and COTS An example of model: COCOTS Some Concluding Remarks

5. Component-Based Software Component-based systems start from the observation that parts of large software systems reappear with regularity The main objectives are those of: • Building large software systems by integrating previously existing software components • Enhance modularity, flexibility and maintainability of systems

6. Component-Based Software Component-based systems could be: • Internally produced software components (programs, libraries, …) • Commercial-off-the-shelf (COTS) products • Components acquired through other means, such as NonDevelopmental Items - NDIs - (e.g. developed for/from governmental agencies) • A mix of all these kind of components

7. Using Components Developing component-based systems is becoming feasible due to the following: • Increase in the quality and variety of COTS products • Economic pressures to reduce system development and maintenance costs • Emergence of component integration technology (see for example Eclipse framework and similar initiatives) • Increasing amount of existing software in organizationsthat can be reused in new systems Components shifts the emphasis from programming software to composing software systems

8. Impact on SW develop. process • Requirements • The set of requirements for the development of the component have been different with respect to the set of requirements of the project (so, possible conflicts arise) • Risk analysis • Risks are also related to external components • Planning of project activities • Difficulties in planning the design and development phases in particular those related to the development of prototypes • Architecture • Selection of standards and components needs to have a sound architectural foundation; this becomes the foundation for system evolution • Development • Integrationphase must be planned early with respect to the late implementation phase • Need to trace the trade-offs made among components during the development process

9. Spiral SW develop. process [Boehm2000]

10. Impact on SW develop. process [Boehm2000]

11. Impact on Integration phase [Boehm2000]

12. Impact on SW develop. Process (2) • Component interfaces standards • Interface standards evaluation of the components are a key part of the process; the degree to which a software component meets certain standards can greatly influence the system interoperability and portability • Reuse of existing components • Reengineering needed for some of the components • External dependences/vendor-driven upgrade problem • An organization can loose part of the autonomy and acquires new dependences when adopting components that are produced externally • System evolution • The evolution of the system is not a simple plug-and-play; side effects can arise after the change/evolution of components

13. The Commercial-Off-The-Shelf components

14. COTS The term “COTS” is meant to refer to components that one can buy, ready-made, from some manufacturers’ • it exists a-priori • it is available to the general public • it can be bought (or leased or licensed)

15. Costs for COTS The true cost of integrating a COTS software component into a larger system includes: traditional costs associated with new software development such as: • requirements definition, design, code, test, and software maintenance + cost of integration: • the cost of licensingand redistribution rights and royalties • effort needed to understand the COTS software • pre-integration assessment and evaluation • post-integration certification of compliance with mission critical or safety critical requirements • Indemnification against faults or damage caused by vendor supplied components • costs incurred due to incompatibilities with other needed software and/or hardware

16. Trade-off in using COTS + The time for development of the whole system, in general, decreases + The cost for the acquisition of the components is, in general, lower than the cost of development + Lack of internal competences + Many possible alternatives to choose But … • Cost of software component integration generally increases • There will be additional system costs to negotiate, manage, and track licenses to ensure uninterrupted operation of the system • Risks quite different from those associated with software developed in-house (so, it requires new methods to mitigate them) • Many possible alternatives to choose

17. Adopting COTS Alternatives Selection/ documentation Alternatives identification Alternatives meets criteria? (“use” of the components) Alternatives Cost/Value Number of alternative components Time [Kontio ‘95]

18. COTS Software Five Panel Model Update Phase Assembly Phase Adapt Phase Quality Phase Market Phase COTS Five Panel Model: turning a set of COTS components in a COTS-based system [SEI99] EVOLUTION

19. Market Phase • Market survey and analysis activities • Dimensions to consider: • Business • Reliability and cooperation of vendors • Negotiation (distinguish between essential requirements and those that can be negotiated) • Components of the cost of the COTS • Technical (“Our biggest problem was with the vendors, especially with believing their claims. Very few components worked as advertised.”) • Operational demonstrations with vendors and users • Distinguish between essential requirements and those that can be negotiated

20. Qualify Phase (1) Investigate the hidden interfaces and other characteristics and features of the candidate products “Component qualification is a process of determining fitness for use of previously-developed components that are being applied in a new system context. Qualification of a component can also extend to include qualification of the development process used to create and maintain it” [SEI document]

21. Qualify Phase (2) Two aspects: • Discovery: the properties of a component are identified • Component functionalities (e.g. service provided) • Component interfaces (e.g. use of standards) • Quality aspects (e.g. reliability, usability, predictability) • Non-technical (e.g. vendors market share) • Evaluation: general criteria for choosing among products (ISO,IEEE) • Paper-based studies • Discussion with other component users • Hands-on benchmarking

22. Adaptation Phase (1) • The component starts its life cycle as a partially developed component • The components are written to meet different requirements and are based on different assumptions with respect to the context of the current project => Components must be adapted in order to minimize conflicts with other components

23. Adaptation Phase (2) Three possible situations: • White boxaccess to source code gives the possibility to rewrite parts of the component code • cost in development, maintenance and evolution - major flexibility • Grey box the component provides its own extension language or API • wrapping, bridging, “glue code” • Black boxonly a binary executable is available and no extension language • wrapping, bridging, “glue code”

24. Assembly Phase (1) Integration of the software components into a software architectural infrastructure (the infrastructure supports assembly and coordination, not simply a “glue”) Several possible architectural styles: • database, in which centralized control of all operational data is the key to all information sharing among components in the system • blackboard, in which data sharing among components is opportunistic, involving reduced levels of system overhead • message bus, in which components have separate data stores coordinated through messages announcing exchanges among components

25. Assembly Phase (2) Some possible problems in this phase: Difficulties during the first integration • Obtain a detail understanding of the components • Modify and Test components • Lack of Documentation (if required) Integration of new versions during developments • Evaluate Cost • Propose integration strategy

26. Update Phase The new versions of components are introduced replacing the old components (volatility of COTS components) Internal activities in response to the changes: • Possible changes in the “glue code” • New tests for the component and for the integrated system (to discover possible side-effects) Two possible external problems: • Components Features evolution • Components Price changes

27. A real project: some important dimensions

28. An example from a real project A set of requirements for a railway system expressed in Natural language in .doc format Need of formalizing the set of requirements in UML and than validating the requirements using a model checker A possible validation process: • Requirements specification/classification • UML description of the set of requirements • UML language transformation in a formal language • Requirements model validation via Model Checking Tecniques

29. An example from a real project Possible basic ingredients for a tool supporting the process: - A requirement management component - A UML modelling component - A Model Checker Huge effort in producing the system from scratch

30. An example from a real project Internal developed libraries Glue Code COTS Word Rational Soft. Architect Requisite Pro Glue code/new functions Glue code/new functions NuSMV Model checker ECLIPSE Exploitation of Eclipse plug-in mechanism (a sort of glue mechanism that simplifies the job related to integration) Languages: Java, C, Lisp To produce the two Glue code/new functions have been developed: - ~ 75.000 Lines of Code (fluctuations during the project related to code refinements) - 5 persons - 12 months

31. Evaluating Costs

32. Costs of Components The trade-off between costs of the component and the cost for the production of the corresponding software autonomously, is one of the parameters to consider during the evaluation phase In any case the cost of the personnel is the key cost in a software project => Models have been set up to evaluate costs

33. General cost estimation Models The general cost estimation model for software systems can be based on the Number of Lines of Code (LOC) in the system/component The output of the models is, in general, a measure of the effort for the persons working in the software project team and is given in Persons/Month Cost-drivers to consider in the model can be classified in: • Personnel (personnel experienced) • Computer (execution time or storage constraints) • Product (reliable requirements) • Project (sophisticated software tools used)

34. Steps for the estimation of costs General steps for the costs estimation of a software project used also in COnstructive COst MOdel (COCOMO) Estimate the software’s eventual size (e.g. Lines of Code) Revise the estimate by using the cost driver or other scaling factors given by the model Apply the model’s tools to the estimate in step 2 to determine the total effort, activity distribution, …

35. An estimation Model A very general model: PM: number of Persons Month LOC: Lines Of Code c: Constant (from cost drivers) k: Constant (> 1) to express a non linear growth with respect to the Lines Of Code of the project (from cost drivers)

36. COTS cost estimation Models • We have to evaluate the cost related to a given component before adopting it • Many cost models have been proposed in literature • We analyze COCOTS (COnstructive COTS Model) derived from the COCOMO model

37. COCOTS model: ingredients (1) Focusing on the costs related to the assembly phase for integration of the components several cost dimensions have to be considered: • Amount of “glue code” (LOC) that is expected to be needed to integrate a set of COTS products into a software application • The percentage of glue code that will be lost due to breakage (BREAK) during the integration effort • Breakage refers to COTS integration code that must be reworked\changed as a result of a change in system requirements or new release by the vendor • The estimated average labor rate (Euro/PM)

38. COCOTS model: ingredients (2) Cost driversparameters also said13 Effort Multipliers (EM)that assumea rating on the given scale from very low to very high, characterizing the conditions pertaining to the COTS integration effort Personnel Drivers 1) ACIEP - COTS Integrator Personnel Experience with Product 2) ACIPC - COTS Integrator Personnel Capability 3) AXCIP - Integrator Experience with COTS Integration Processes 4) APCON - Integrator Personnel Continuity (Stability) COTS Component Drivers 5) ACPMT - COTS Product Maturity 6) ACSEW - COTS Supplier Product Extension Willingness 7) APCPX - COTS Product Interface Complexity 8) ACPPS - COTS Supplier Product Support 9) ACPTD - COTS Supplier Provided Training and Documentation Application/System Drivers 10) ACREL - Constraints on Application System/Subsystem Reliability 11) AACPX - Application Interface Complexity 12) ACPER - Constraints on COTS Technical Performance 13) ASPRT - Application System Portability

39. COCOTS model: ingredients (2) Cost driversparameters also said13 Effort Multipliers (EM)that assumea rating on the given scale from very low to very high, characterizing the conditions pertaining to the COTS integration effort Personnel Drivers 1) ACIEP - COTS Integrator Personnel Experience with Product 2) ACIPC - COTS Integrator Personnel Capability 3) AXCIP - Integrator Experience with COTS Integration Processes 4) APCON - Integrator Personnel Continuity (Stability) COTS Component Drivers 5) ACPMT - COTS Product Maturity 6) ACSEW - COTS Supplier Product Extension Willingness 7) APCPX - COTS Product Interface Complexity 8) ACPPS - COTS Supplier Product Support 9) ACPTD - COTS Supplier Provided Training and Documentation Our Application/System Drivers 10) ACREL - Constraints on Application System/Subsystem Reliability 11) AACPX - Application Interface Complexity 12) ACPER - Constraints on COTS Technical Performance 13) ASPRT - Application System Portability

40. COCOTS model: some details 1) ACIEP - COTS Integrator Personnel Experience with Product How much experience did/does the development staff have with running, integrating, and maintaining the COTS products? Metric: months/years of experience with product Very Low (2): staff on average has no experience with the products Low (1.5): staff on average has less than 6 months' experience with the products Nominal (1.0): staff on average has between 6 months' and 1 year's experience with the products High (0.66): Staff on average has between 1 and 2 years’ experience with the products Very High (0.5): Staff on average has more than 2 years' experience with the products

41. COCOTS model: process • Estimate the amount of “glue code”(LOC) • Estimate the percentage of “glue code” that will be lost due to breakage(BREAK) • Determine the effective size (SIZE) of the glue code development effort by feeding the estimates derived in steps 1 and 2 into the formula for SIZE • Assign each effort multiplier (EM) a rating on the given scale from very low to very high • Determine the overall estimated effort (PM) for this integration task by feeding the estimate for SIZE and the rated effort multipliers into the formula for Person-months (PM) • Determine theestimated cost by multiplying estimated PM by the estimated average labor rate (Euro/PM)

42. COCOTS model: formulas BREAK: Breakage: Percentage of COTS glue code thrown away due to requirements volatility LOC: Size of COTS component “glue code” expressed in (thousands of source) Lines Of Code PM: Number of Person Months of estimated COTS integration effort Euro/PM: Estimated average labor rate per person-month EM: The thirteen Effort Multipliers or cost drivers, each of which assumes one of five possible values based upon the following ratings: very low, low, nominal, high, and very high (nominal ratings have a multiplier value of 1.0)

43. Cost of assembly for the project • Duration: 12 months • Line Of Code: ~80.000 • BREAK: ~ 15% • ACIEP: .66 (product of all the effort multipliers .00078 for the project) Size = 80000*(1+0.15) = 92000 PM = (1/12)*92000*(.66*…*…) => (1/12)*7660*0.00078 => ~6 persons month Assembly_cost = 2800 Euro/month * 6 persons = 16800 Euro/month

44. Total costs for a project Total_project_COST = Product_licenses_COST + (Cost/price of the product) Qualify_COST + (Filtering effort and number of candidates) Adapt_COST + (Effort to tailor COTS to meet requirements) Assembly_COST (Integration effort and code breakage)

45. Concluding Remarks

46. Risks and Lessons Learned Some remarks from Federal Aviation Administration - Software Engineering Resource Center • Distinguish between essential requirements and those that can be negotiated; successful use of COTS solutions requires the capability to modify requirements • Keep requirements negotiable until the system’s architecture and COTS choices stabilize • Skill level and experience are important; this includes people on the acquirer’s side who are determining essential requirements as well as the COTS integrators • Verify COTS with operational demonstrations; bring the users into the operational demonstrations • Establish a technology watch to track vendors and products • Use mature products for safety-critical applications • Expect to spend time in training. In choosing a system integrator, look not only at experience in the application domain but also with COTS integration in general and with the specific products to be integrated

47. Risks and Lessons Learned Some remarks on the process • Use risk management and risk-driven spiral-type process models • Assess risks via prototyping, benchmarking, reference checking, and related techniques • Focus each spiral cycle on resolving the most critical risks. • Perform a “receiving inspection” upon initial COTS receipt to ensure that the COTS product really does what it is expected to do

48. References Software Engineering Institute (SEI) - Carnegie Mellon University (CMU) http://www.sei.cmu.edu Center for Systems and Software Engineering (CSSE) - University of Southern California (USC) http://csse.usc.edu/csse/ [Boehm2000] Barry Boehm. Spiral Development:Experience, Principles, and Refinements. Spiral Development Workshop,edited by Wilfred J. Hansen, CMU, 2000. [Kontio95] Jyrki Kontio.A Case Study in Applying a Systematic Method for COTS Selection. Proceedings of ICSE ‘96. [szyperski98] Clements Szyperski, Component Software Beyond Object Oriented Programming, Addison-Wesley, 1998.