Outline

2. Outline • Why COTS? • Perceived Benefits • COTS Environments • COTS Interoperability • Is Software Like Hardware? • Observations of Successful COTS • Criteria for Using COTS • Problems With Using COTS • Reliability Considerations • Maintainability Considerations • Network Considerations • Personnel Implications • Life Cycle Models • COTS Checklist • Questions for Audience

3. Why COTS? • The appeal of COTS: • Another in a long line of silver bullets. • Remember Structured Design and Programming, Ada? • DoD has embraced COTS. • Has the issue been entirely thought through? • The COTS bandwagon makes sense for office products (e.g., MS Office) because the criteria for successful application are satisfied (e.g., standard functions).

4. Why COTS?(continued) • Direct application to other user areas (e.g., safety critical) is problematic. • Three dimensions of COTS are crucial to successful application: • Technical: degree of integration and interoperability achieved within the larger system. • Managerial: infrastructure for the certification of COTS in given applications. • Resource: library of certified components. • Internal, Web, vendor-supplied.

5. Perceived Benefits • Reduced design. • Reduced programming. • Reduced cost. • Reduced testing. • Reduced time to market. • Reduced risk. • Increased reuse across multiple applications. • Increased interoperability.

6. Perceived Benefits(continued) • Must distinguish between vendor and user application of COTS: • Many vendors produce products that are not domain specific (e.g., network server) and have limited functionality (e.g., mobile phone). • Many users develop systems that are domain specific (e.g., target tracking system) and have great variability in functionality (e.g., corporate information system).

7. Perceived Benefits(continued) • COTS should benefit vendors directly in terms of the components they acquire and integrate into their products. • If cost savings passed to users for their COTS components, they would benefit indirectly. • How would the parts of user systems that are domain specific benefit from COTS? • User could use components-based design paradigm but much of today’s software is embedded in legacy systems -- difficult to redesign. • The following diagram shows the situation:

8. User Software System COTS NON-COTS Network Server Target Tracking System Not Domain Specific Domain Specific

9. COTS Environments • Important to distinguish environments in which COTS can operate -- some environments are easy, others are hard. • Office products: spreadsheet. • Easy: Many people use these standardfunctions. • Vendors’ products: communication protocol. • Relatively easy: High volume, limited functionality. • JAVA produced components.

10. COTS Environments(continued) • User-developed applications like an MIS or Space Shuttle. • Hard: Functions are domain specific. • No COTS available. • Even if COTS were available, it might be unwise to use it because of risk and reliability concerns.

11. COTS Interoperability • Even in the “easy” environment of office products interacting with the larger system, there is limited interoperability: • Certain data can be imported and exported. • Certain objects like documents and drawings can be imported and exported after a fashion. • Program objects (e.g., equations from a math or statistics program) cannot be imported into a spreadsheet program.

12. Is Software Like Hardware? • The following table shows a comparison of hardware -- the ultimate “COTS” -- with software with respect to the concept of interchangeable and replacement parts. • What will be system availability using COTS? • Unlike hardware, availability cannot be kept high by “replacing” the software. • Fault tolerant software is a possibility but it has had limited success.

14. Observations of Successful COTS • Operating Systems • Word Processors • Spreadsheets • Graphics • DBMS • Utilities • Communication Protocols

15. Criteria for Using COTS • Lessons learned from observation of successful COTS applications: • Generality of application (i.e., not domain specific). • Large-scale use. • Users perform functions in a standard way: • Point and click, copy and paste.

16. Criteria for Using COTS(continued) • Ease of integration. • Must “fit” existing application. • Spreadsheet is very popular because it mimics the way people do their work. • Simulation of the desktop is very popular because it creates a familiar environment.

17. Criteria for Using COTS(continued) • Appropriate for users to operate in a standard way. • Save money. • Economic considerations outweigh restrictions in flexibility (e.g., spreadsheet format rules) • Communicate with other parts of the system. • Communicate with products in same office suite.

18. Problems With Using COTS • Loss of control. • Development and maintenance no longer done by the user but still responsible for results. • Loss of domain knowledge. • User no longer developing applications. • Increased dependence on vendor knowledge and support. • Consequences of vendor no longer supporting product.

19. Reliability Considerations • Fundamental problem: A piece of software will exhibit different reliability performance in different applications and environments. • A COTS component may have a favorable reliability rating when operated in isolation but a poor one when integrated in a larger system. • What is needed is the operational profile of COTS as integrated into the larger system.

20. Reliability Considerations(continued) • How do we estimate the reliability of COTS when there is no data available from the vendor? • How do we estimate the reliability of COTS when it is embedded in a larger system? • How do we revise our reliability estimates once COTS has been upgraded?

21. Maintainability Considerations • Suppose a bug occurs in an embedded COTS system: • Is it in COTS or the larger system? • Suppose it is caused by an interaction of the two. • The user knows the bug has occurred, but does not know how to fix it. • The vendor does not know the bug has occurred: • Does not know how to fix it. • Does not know the larger system.

22. Maintainability Considerations(continued) • Suppose the user needs to upgrade the larger system and this upgrade is incompatible • How are required enhancements to be installed? • The user will be forced to upgrade COTS with a new release whether it it needed or not.

23. Network Considerations • Integration of software on a network presents a formidable challenge to installers, absent COTS. With COTS, life will be even more “interesting”. • To make it possible to use COTS products on a network, vendors must write their software for network environments. • How can this be ensured?

24. Personnel Implications • More emphasis on acquisition and less on development. • More emphasis on assembly and integration and less on construction. • More knowledge of vendor products. • More knowledge of cost-benefit analysis. • Less knowledge of programming. • The following diagram shows differences between traditional and COTS life cycles:

25. Output: O-O Generic: Objects. Relationships. Operations. O-O Compiler Output Test Objects, Relationships, & Operations. Object Failures. Inspections IV&V SRE SRE Traditional Life Cycle Phase Requirements Analysis Design Programming Test Operations & Maintenance Requirements Test Plan. Discrepancy Reports. Change Requests. Discrepancy Reports. Specifications Listing Functions Change Objects, Relationships, & Operations. Object Failures. COTS Life Cycle Phase Component-Based System Design Operations & Maintenance of Component-Based Systems Requirements Analysis & Acquisition of Components Assembly of Components Integration of Components & System Test Key Process Areas: Reuse

26. COTS Checklist • Application Domain. • Reliability, Maintainability, Availability. • Compatibility, Integration, Interoperability. • Security. • Hardware, Operating System, Network Capability. • Library resources. • Test results under the operational profile of user. • Certification, Warranties. • Documentation, Track Record, Support Service.

27. Summary • The decision to employ COTS boils down to one of control: • Will the user be at the mercy of the vendor? • COTS is highly desirable if: • the user remains in control. • the conditions implied by the checklist are satisfied. • The applicability of COTS is less than advertised because much of what software does is domain specific.

28. Questions for Audience • Any operational COTS applications? • What are their characteristics? • What custom-made software did it replace? • When did it happen? • What are your candidate applications? • What are their characteristics? • When will it happen? • What custom-made software will it replace?