1 / 21

Jim Buffington

Validation & Verification of Intelligent and Adaptive Control Systems (VVIACS). Jim Buffington. Outline. Introduction Motivation Approach Assessment Development Evaluation Summary. Team. Vince Crum – AFRL - Government PM Jim Buffington – LM Aero - Contractor PM

fairly
Download Presentation

Jim Buffington

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validation & Verification of Intelligent and Adaptive Control Systems (VVIACS) Jim Buffington

  2. Outline • Introduction • Motivation • Approach • Assessment • Development • Evaluation • Summary

  3. Team Vince Crum – AFRL - Government PM Jim Buffington – LM Aero - Contractor PM LM Aero – Greg Tallant, Peter Stanfill LM M&FC - Clinton Plaisted, Barry Frazier, Rich Hull, Guy Rowlands LM SS - Prasanta Bose Carnegie Mellon University - Bruce Krogh General Electric Global Research - Tim Johnson, Hunt Sutherland Scientific Systems Company, Inc - Ravi Prasanth, Sanjeev Seereeram

  4. Scope • Safety-Critical Flight Systems • Military Certification • Advanced Controls • System Development • All Development Phases • Process, Tools, and Methods • Verification & Validation

  5. Emerging Software Size and Complexity Projected Exponential Increase in SW Size and Complexity • Advanced system attributes (on-board intelligence and adaptive control laws) will be required to accommodate emerging functional requirements. • This will increase the size and complexity of control systems beyond the capability of current V&V practices. Inter-System Communication & Dependencies UAV - Projected • Increasing system integration requirements and Complexities F-35 – Est JSF CDA F-22 YF-22 IDF Block 60 F-16

  6. Emerging System Costs Size and Complexity Increase Costs • SW size and system complexity lead to growth system development and certification costs • Test automation insufficient for emerging control systems

  7. Compressed Development Schedules Emerging Program Requirements Increase Risk • Customers requiring 12 to 36 month development schedules • Baseline is 48 months to first flight

  8. Purpose GOAL: Enable affordable development of future safety-critical flight systems with prescribed levels of safety and reliability. OBJECTIVE: Study, develop, and demonstrate effective V&V technologies for advanced safety-critical control system flight certification. • Classify emerging safety-critical control systems according to fundamental attributes • Develop and demonstrate preliminary V&V strategies that focus on critical flight certification schedule and cost points • Identify high-payoff V&V process, tool, and method technologies for further development APPROACH: • Use extensive experience-base and diverse team to develop innovative concepts • Evaluate concepts in a realistic framework to maximize transition success

  9. Tasks • Task 1 • Emerging Control System Study • Task 2 • Control Characteristics and V&V Needs Study • Task 3 • Innovative Flight Certification Strategies Development • Task 4 • Proof of Concept • Task 5 • Technology Development Planning and Reporting 1. Assessment 2. Development 3. Evaluation

  10. Tasks 1 & 2 – Assessment • Emerging Control Systems Study • Develop control system project database • Down select to subset of projects for additional analysis • Control Characteristics and V&V Needs Study • Analysis to define emerging fundamental properties • Identification of V&V drivers • System Development Model • System Development impact assessment V&V NEEDS

  11. Task 1 – Emerging Control Systems ECS PROJECT AIMSAFE / RESTORE ICARUS PCA LOCAAS Swarm Intelligence XACT Software Enabled Control Autonomous Propulsion System Tech Intelligent Engine PHM Distributed Space Systems DESCRIPTION Integrated Management, Adaptive Control Intelligent Autonomy Polymorphous Computing Architectures Autonomous Control Bio-inspired Multi-vehicle Control Adaptive Failure Management Optimal Trajectory Generation Intelligent Failure Management Model-based Health Management Distributed Multi-vehicle Control Characterized 10 Emerging Control System (ECS) projects • 6 ECS projects used to form the Single-Vehicle ECS (AIMSAFE/RESTORE, PCA, LOCAAS, XACT, Autonomous Propulsion System, and Intelligent Engine PHM) • 5 ECS projects used to form the Multi-Vehicle ECS (ICARUS, PCA, Swarm Intelligence, Software Enabled Control, and Distributed Space Systems)

  12. Task 2 - Emerging Systems Analysis Extensible Analysis Framework based on Fundamental Properties • Identified 100+ fundamental properties (FPs) for the ECS projects • Categorized FPs in 5 Views: • Requirements, System, Algorithm, SW, HW • Identified 28 FPs as Emerging Fundamental Properties (EFPs) • Identified 4 primary V&V Drivers • Difficulty, Complexity, Unfamiliarity, External factors • System development model • Based on LM Process • Multiple levels • Contains over 800 tasks • MS Project / Excel

  13. Task 2 - Impact Analysis Results Significant Cost/Schedule Increase Projected Due to Complexity • Single-Vehicle ECS Increases Development Costs ~ 50%, V&V Costs ~ 100%, and Critical Path Length ~ 50% • Multiple-Vehicle ECS Increases Development Costs ~ 100%, V&V Costs ~ 150%, and Critical Path Length ~ 125% • Software: Single-Vehicle 100% Increase and Multiple-Vehicle 200% Increase in V&V Costs • Test: Single-Vehicle 150% Increase and Multiple-Vehicle 250% Increase in V&V Costs

  14. Task 2 - Pareto Analysis Pareto Analysis Identifies the Critical Functional Disciplines • Single-Vehicle ECS V&V Cost Impact • 80%: TEST, SW • 90%: 80% + TTD • TEST largest component (41%) followed by SW (28%) • Single-Vehicle ECS V&V Duration Impact • 80%: SW, TEST • 90%: 80% + TTD • SW dominates at 53%

  15. Task 3 – Flight Certification Strategies Development • Requirements Development • Requirements/Guidelines • V&V Drivers Mapping • Flight Certification Refinement • Current Process Development • Evolution of Process Development for V&V • V&V Methods Development • Identify Current V&V Methods • Recommend Approaches V&V STRATEGY EVOLUTIONS

  16. Task 3 – Strategy Evolutions Near-Term (1-3 yrs) Evolution: System Model-based design now being implemented • Auto-Code • Auto-Test • Rapid Prototyping • System Model-Based • Automated Verification Management • Simulation-Based design Mid-Term (4-6 yrs) Evolution: Formal Foundations in advanced development • Formal Requirements Specs • Requirements and Traceability Analysis • Formal Methods • Computer-Aided System Engineering Far-Term (7-9 yrs) Evolution: V&V Awareness throughout  still in research • Run-Time V&V • Rigorous Analysis for Test Reduction • Requirements & Design Abstraction • Probabilistic/Statistical Test • Testing Metrics

  17. Task 4 – Advanced Technology Impact to System Development • System Development Cost Reductions: • Baseline: 25% • Single-Vehicle: 33% • Multi-Vehicle: 35% • System Development Critical Path Reductions: • Baseline: 12% • Single-Vehicle: 29% • Multi-Vehicle: 30%

  18. Task 5 – Technology Development Planning and Reporting • Risk Waterfall Planning  ROM Development Cost • EFP Coverage Analysis + Task 4 System Development Impact  Net Benefit • Priority of V&V technologies was established on the basis of a cost-benefit analysis performed on each technology • Cost-Benefit Ratio (CBR) = ROM Development Cost/Net Benefit • All near-term technologies except Automated Verification Management were eliminated from the CBR analysis because these technologies are relatively mature (moderate technical risk) and significant industry investment in these technologies is ongoing and is expected to continue.

  19. Task 5 – Technology Cost / Benefit • ROM Cost – Risk Waterfall Analysis • NetBenefit is defined as the product of the technology’s EFP CoverageBenefit and the System DevelopmentBenefit • BNET = BEFPBSD • EFP Coverage Benefit is a measure of the technology’s applicability across the set of EFPs (large BEFP implies good coverage) • System Development Benefit is a measure of the technology’s impact on system development cost and schedule

  20. Task 5 – Final Prioritized Technologies Based on Cost-Benefit Analysis • Automated Verification Management • Formal Requirements Specifications • Requirements and Traceability Analysis • Formal Methods • Probabilistic / Statistical Test • Requirements and Design Abstraction • Run-Time V&V • Testing Metrics • Rigorous Analysis for Test Reduction • Computer-Aided System Engineering Increasing Cost-Benefit Ratio • EFP weighting: • WEFP = 1 (all equal) • ECS weighting: • 10% Baseline • 50% Single-vehicle • 40% Multi-vehicle Near-Term (1-3 years) Technology Mid-Term (4-6 years) Technology Far-Term (7-9 years) Technology

  21. Summary • Developed emerging control system (ECS) database and populated with an extensive set of past, present, and future ECS projects. • Assessed impact of future systems on current development process and identified “long-pole” functional disciplines with highest adverse impact (i.e., SW, TEST, TTD). • Identified 15 key V&V technologies to address system development impact and characterized in terms of near- (1-3 years), mid- (4-6 years), and far-term (7-9 years) strategy evolutions. • Demonstrated effectiveness of each strategy evolution using the system development model and the system development impact assessment tool (i.e., cumulative 25% reduction in V&V cost and 12% reduction in V&V effort for all strategies). • Developed technology maturation plans for each V&V technology identified, and prioritized the technologies by performing a cost-benefit analysis.

More Related