1 / 19

Independent Verification and Validation Current Status, Challenges, and Research Opportunities

Independent Verification and Validation Current Status, Challenges, and Research Opportunities. Dan McCaugherty IV&V Program Manager Titan Systems Corporation 5 September, 2001. Agenda. Where we are today Developer challenges observed by IV&V IV&V challenges Opportunities for R&D.

montana
Download Presentation

Independent Verification and Validation Current Status, Challenges, and Research Opportunities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Independent Verification and ValidationCurrent Status, Challenges, and Research Opportunities Dan McCaugherty IV&V Program Manager Titan Systems Corporation 5 September, 2001

  2. Agenda • Where we are today • Developer challenges observed by IV&V • IV&V challenges • Opportunities for R&D

  3. Where we are today

  4. IV&V Gaining Acceptance • IV&V is gaining acceptance as a mechanism for mitigating software risk • IV&V is naturally evolving to also include programmatic assessments such as cost/schedule performance analysis • Recent NASA IV&V policy presents an opportunity for wider application and understanding of IV&V

  5. IT Project Success Factors • Gartner Institute study identified the following essential areas as critical to IT project success1 • Risk Management • Communications • Resource Management • Lessons Learned • Additional areas based on professional experience • Project planning • Requirements!!! • Projected that implementation of sound project management approach emphasizing these key areas can reduce number of failed projects by 30%1 1GartnerGroup

  6. IT Project Success Rates • Odds for Success1&2 • > 80% of IT projects are late, over budget, lack essential functionality, or are never delivered • 28% of IT projects just flat fail representing more than $81 Billion in losses • Of 15,000 data migration projects started in 1999, as many as 88 percent either overran or failed • Leading causes for project failure1 • poor project planning • failure to manage scope and change • failure to manage risk 1GartnerGroup 2The Standish Group

  7. Developer Challenges Observed by IV&V

  8. Incremental Delivery Definition • “Build a Little, Test a Little” approach can avoid major development failures, but creates other challenges, such as: • Maintaining end state specifications concurrent with increment specifications • Implementing the right requirements in the right builds to meet testable increment specific capabilities • Understanding the cost/schedule effects of pushing requirements into later builds

  9. Early Definition of Interfaces and Key Architectural Concepts • Late interface or architecture changes or problems create significant re-work or are unmanageable • Distributed computing stresses system engineering efforts for early definition of • API’s, remote procedures, and reused objects • Redundancy Management, • Fault Detection, Isolation, and Recovery • Exception handling

  10. Use of New Technologies • A great way to attract and keep highly skilled professionals • Majority of focus on what the technology can do, not enough focus on what it can not do or was never designed to do • Limited skill base for new technologies • If COTS, may not have future support

  11. Measuring Progress or Quality • Measurable definitions for “Done” are needed. • Metrics efforts and collection mechanisms need to be part of the process from day one • Optimism (a good thing) must be tempered with facts

  12. IV&V Challenges

  13. IV&V Technical Challenges • Adapting IV&V methods to developer environments • Understanding system dynamic behavior • Convincing a project that the process is too risky • Validating Non-Development-Items • Reverse engineering incompletely specified systems

  14. IV&V Managerial and Business Challenges • Demonstrating that selected IV&V tasks and scope are appropriate for the Project’s acceptable levels of risk • Appropriate IV&V integration into project • Demonstrating IV&V ROI • Accounting for IV&V data needs within developer contracts.

  15. Opportunities for R&D

  16. Opportunities for R&D • Research IV&V ROI • Past case study3 shows 1.25 < IV&V ROI < 1.82 • Recommend a study of many projects with varying amounts and types of IV&V • Correlate overall project success to IV&V effort types • Continue development of code reverse engineering tools and interface V&V tools • Software and Interface Analysis Tool (SIAT) • Interface Management System (IMS) • Develop Reconfigurable engineering simulation for Space Craft • Improve early “validation” analyses 3Rogers

  17. Opportunities for R&D • Develop a guidebook identifying sets of IV&V tasks for certain types of systems and acceptable levels of mission risk. • Research history of IV&V method application and results • Develop a guidebook for developer managers on how to realize benefits of IV&V. • Contracts and agreements • Integration of IV&V into different types of projects • Reporting and issue resolution

  18. Summary

  19. Summary • There are many software challenges/risks out there • Many can be managed through use of accepted engineering practices • IV&V can help mitigate risk if used/applied correctly • We need to better quantify the effects of IV&V and the ROI • Recommended areas of R&D • Research IV&V ROI, effects of IV&V use • Develop tools that are needed today • Develop guidance for future IV&V'ers and IV&V users.

More Related