True Program Success TM

1. True Program SuccessTM Costs of Security in a COTS-Based Software System Arlene Minkiewicz, Chief Scientist PRICE Systems, L.L.C. arlene.minkiewicz@pricesystems.com October 2004

2. Conclusion • COTS solutions can save time and money in the development and life-cycle phases of a software product. • Security constraints on a software system may impact the cost/benefit analysis when building a COTS-Based software system • Understanding of issues associated with COTS systems and security implications is essential to successful deployment of COTS-Based Software Systems

3. Cost of security in a COTS-Based Software System • The Problem • Solution Methodology • Security Issues • Six Steps to a Successful COTS Implementation • Adding security to the six steps • Conclusions

4. The Problem • COTS - Not always the low cost solution • Adding security constraints to a software system will change the factors involved in a cost / benefit analysis comparing COTS solutions to home grown solutions • Need to understand all of the activities associated with a COTS based solution in order to properly assess the cost of a COTS Based solution • Need to understand the impact of security constraints on the costs of these activities

5. Solution Methodology • Understand and bound the problem • Understand the process of including COTS and identify the activities associated with this process • Identify the factors that drive costs for these activities • Identify the impact of security constraints on the costs of these activities • Construct a mathematical model to determine cost from these cost drivers • Test the mathematical model against actual data and refine the model

6. Bounding the Problem • Extended Definition of COTS Product (from USC CSE) to include modifications • Commercially available software product - sold, leased or licensed • Source code sometimes unavailable • Periodic release with new features, upgrades for technology, etc. • Modifications to software • Focus is on COTS products being embedded in new software systems • Additional focus is on those systems with security constraints • Security constraints defined as acceptance criteria related to Evaluation Assurance Levels as outlined in the Common Criteria for IT Security Evaluations

7. Security Requirements Present in Two Forms • Additional functional requirements related specifically to security related features • Encryption algorithms • Password protection • Remote access security procedures • Additional levels of qualification and testing to ensure that the software does not allow security breaches into the system on which it operates • Backdoors • Buffer overflows • Other defects that allow entrée to hackers • Patches that can be reversed engineered to find weaknesses

8. Cost Impacts of Security Requirements • Additional Functional Requirements Related to Security increase total Functional Size of the software (SLOC, Function Points, etc.) • Impact of Assurance Requirements on Cost are a function of: • COTS Selection Strategy • Process Maturity of Organization Delivering Solution • Focus of process model on security • Expertise of personnel relating to Secure Software Development and Good Software Engineering practices

9. COTS Selection Strategy • Once a decision has been made to incorporate COTS components into a system with security constraints, the integrator has to pick a strategy • Buy and wrap • Select components that best meet overall functional requirements • Develop a wrapper with glue code that encapsulates the COTS components, ensuring that security requirements are met • Buy pre-certified components • Only evaluate COTS components with vendor certification at the required Evaluation Assurance Level • Buy and certify internally • Select components that best meet overall functional requirements with vendor assurance that they comply with security requirements • Perform necessary certifications internally

10. Process Maturity of Organization • Biggest area for security concerns is in the quality of the software • Organization with a good software process model firmly entrenched build higher quality software • Processes focused on security get the best results • Cleanroom • Formal mathematical methods • Good processes in general still make substantial difference • CMI • PSP/TSP • Good software development practices can substantially reduce impact of security assurance requirements because most security threats arise from the presence of defects in design and implementation

11. Expertise of personnel • Security Assurance Requirements are most likely to be met if security is designed into the software from the very beginning • Personnel with training and / or experience in the development of secure software systems understand this • Personnel with training and / or experience in good software development practices understand the importance of building quality into the process from the very beginning

12. Six Steps to a Successful COTS Implementation • Analyze Software Requirements • Evaluate and Select COTS Solution(s) • Negotiate terms with the COTS Vendors • Implement COTS Based Solution • Tailoring • Modifications (not good but sometimes deemed necessary) • Develop Glue Code • Integration with other COTS Components or homegrown components • Maintain License, Subscription and Royalty fees • Maintain and Upgrade COTS-Based Solutions

13. Analyze Software Requirements • Necessary whether software is being built or bought • In fact part of the requirements discussion should be whether building or buying makes sense • Selection criteria should relate back to requirements • Care should be taken to identify where there is flexibility – as no COTS solution will meet all software requirements completely • COTS Selection Strategy decided during Requirements Analysis • Care should be taken to understand process maturity and personnel capability with respect to secure software development when selecting the best strategy for success

14. Identify, Evaluate, and Select • Identify solutions that satisfy product, vendor and security requirements • Techniques for evaluation include • progressive filtering • puzzle approach • keystone components • COTS Selection Strategy is key factor in this activity • Buy and wrap – not an issue • Buy pre-certified or Buy and Certify – clearly will impact evaluation process

15. Negotiate terms with COTS vendors • Understand that vendor cooperation and forthrightness is best during the negotiation phase • Address and resolve missing or incomplete functional and known bugs before signing on the dotted line • Establish expectations for responsiveness to issues identified once the integration effort has begun • Develop a clear picture of the recurring and non-recurring costs of the system being developed • Security issues impact negotiations and costs • Pre-certified components bear the cost of certification and re-certification – understand how that impacts costs. • If components are to be certified by integrating organization – be sure to include provisions in the negotiations in the event certification fails to meet promised assurance levels.

16. Implement the COTS Based Solution • Tailoring includes non development activities that must be applied to the COTS components to meet system requirements. • Modifications sometimes occur • Need to understand impact on cost and crossover where modified COTS cost more than home grown solutions • Buy and Wrap COTS – security is not an issue as wrapper will encapsulate modified component • Modifications would require complete re-certification if component is certified by the vendor or the purchaser

17. Implement COTS Based Solutions • Glue code is code developed to hold all the components of the system together • Wrapper would be considered part of the glue code • Costs for glue code development would be impacted by security requirements • Good processes, training and expertise would mitigate this cost impact • System level integration and tests ensure that all the components function together to meet requirements • Part of integration and test would be assurances that the total system meets all assurance requirements • Costs would be impacted by security requirements • Good processes, training and expertise would mitigate this cost impact

18. Maintain license, subscription and royalty fees • Important to perform a long term analysis to understand the long term external costs of implementing a COTS based solution • Initial negotiations should be used to ensure that certified or promised level of security is maintained with upgrades • Renewal period is a good opportunity to revisit terms of negotiations to determine whether vendor is meeting support and upgrade commitments

19. Maintenance and Upgrade of COTS solutions • Evaluation and possible inclusion of upgrades and updates • Perform evaluation to determine whether upgrade adds value to Software System • Re-perform internal certifications • Modifications to wrapper code if necessary to accommodate new interfaces • Costs for glue code modifications and integration and test will be impacted by security constraints • Good processes, training and experience will mitigate security cost impacts • Fix bugs • In glue code, modifications or to compensate for COTS bugs not fixed by vendor • Costs impacted by security constraints • Reintegration necessary • Good processes, training and experience will mitigate security cost impacts

20. Conclusion • Six Steps to a Successful COTS Implementation • Analyze Software Requirements • Evaluate and Select COTS Solution(s) • Negotiate terms with the COTS Vendors • Implement COTS Based Solution • Tailoring • Modifications (not good but sometimes deemed necessary) • Develop Glue Code • Integration with other COTS Components or homegrown components • Maintain License, Subscription and Royalty fees • Maintain and Upgrade COTS-Based Solutions

21. Conclusion • COTS solutions can save time and money in the development and life-cycle phases of a software product. • Security constraints on a software system may impact the cost/benefit analysis when building a COTS-Based software system • Understanding of issues associated with COTS systems and security implications is essential to successful deployment of COTS-Based Software Systems Arlene F. Minkiewicz Chief Scientist PRICE Systems, L.L.C 856-608-7222 Arlene.minkiewicz@pricesystems.com

22. FASTER DECISIONS. BETTER DECISIONS.

23. Leader in Program Affordability Management solutions Combine cost estimating, project control, and knowledge management – ensuring project success at every decision gateway Customers increase visibility, minimize risk and cost, accelerate project development, and improve the effectiveness of project selection, control and deliverywww.PRICESystems.com About PRICE Systems