Systems Engineering for Automating V&V of Dependable Systems John S. Baras

1. Systems Engineering for Automating V&V of Dependable Systems John S. Baras Institute for Systems Research University of Maryland College Park 301-405-6606 baras@isr.umd.edu NITRD HCSS-AS National Workshop on Aviation Software Systems: Design for Certifiably Dependable Systems October 5-6, 2006 Alexandria, VA

2. Aviation systems are complex heterogeneous engineering systems --hardware and software components Must be viewed as distributed, asynchronous and hybrid dynamic systems Systems of subsystems that sense, make decisions and executeactions ---- many closed-loop subsystems Subsystems that perform this sensing or decision making or action execution are not co-located Communications occur between sensing blocks, decision making blocks and action execution blocks that are subject to greatly varying constraints on timing, communication bandwidth and delay This distributed asynchronous dynamic systems view of avionics systems has not been promoted to date Essential, in our view, for understanding: fundamental architectural issues stability and robustness performance vs complexity trade-offs leads to new fundamental rethinking of the foundations for dynamic collaboration between local subsystems, subject to the constraints of distributed real-time operation, asynchronous operation, bandwidth, delay. Aviation Systems and Software

3. Current and future aviation systems are software intensive systems Furthermore they are net-centric systems -- they involve many interacting and collaborating agents (c.f. systems or subsystems) In any approach to design for certifiable dependable systems, a systems engineering methodology must be followed – means specifically that interactions with human users, other systems and subsystems, and the environment must be accounted for and evaluated Challenges: Architecture Requirements and their Management Formalization of the constraints imposed by the physical layer(s) What is meant by a dependable system, as well by certification of a dependable system is not well understood for systems with the characteristics described above. ASS as Distributed Hybrid Systems

4. We advocate a Compositional Approach to design for certifiable dependable systems Emphasize dynamic systems as well as dynamic dependability (i.e. we include dynamic monitoring, sensing and corrections as allowed means to achieve dependable systems) Approach marries quantitative systems engineering with a compositional approach to networked systems -- Components are the critical elements. Certification involves both hard certifications well as soft certifications and is accomplished by a synergistic application of performance analysis (optimization, constrained based reasoning, logic) as well as formal models (mode checking, automatic theorem proving, timing analysis including concurrency). Our long term approach will utilize: mixture of methods from computer science (distributed communicating processes, formal models, concurrency, formal verification-validation, model checking, automatic theorem proving) and from control-communication systems (hybrid systems, multi-agent systems, feedback, system dynamics and stability, change detection, adaptive control and correction, robustness). Compositional Approach -- Components

5. We develop formal dynamic models for ASS that respect the constraints, while at the same time formally specifying the structure(what the ASS consists of?) and behavior (what the ASS does?) from SE perspective. Within this framework that distributed and asynchronous operation will be built in as constraints (logical or numerical), and where timing, bandwidth and delay constraints between sensing, decision making and action execution blocks will also be modeled. To completely model and understand properties of ASS we need a framework that combines logical and numerical models, thus hybrid systems. But we also need a combination of methods that can handle these hybrid models for decision making, robustness, inference Compositional Approach -- Components

6. Iterate to Find a Feasible Solution / Change as needed Change structure/behavior model as needed Define Requirements Effectiveness Measures Map behavior onto structure Allocate Requirements Create Sequential build & Test Plan Specifications Perform Trade-Off Analysis Assess Available Information Create Behavior Model Create Structure Model Compositional System Synthesis & Integration Model-based Beyond UML Rapsody UPPAAL Artist Tools MATLAB, MAPLE Modelica DOORS, etc OPCAD CPLEX, SOLVER, ILOG Integrated System Synthesis Tools - Environments missing … Generate derivative requirements metrics Model-Based Information-Centric Abstractions Integrated Multiple Views is Hard !

7. From a Reductionist Approach to an Integrative Approach The challenge is to generate system predictable behavior by integrating behaviors of the components It is not all in the software environments Need a combination of Model-Based system and software design and integration and Deeper analysis of system models and properties Compositional System Synthesis and Integration: the Next Frontier

8. Domain Specific Modeling Languages (DSML) with semantics that can be composed and manipulated Composition platforms correct by construction systems platforms and models of computations; substantial reduction in V&V System and component behavioral abstractions that can support Incremental System Integration  while preserving testability and predictability Fully integrated semantically control, software and systems design tools and platforms Model-Based System and Software Design and Integration

9. Principles for system integration System Science  Network Science Fundamental performance limitations of networked systems Fundamental implications of physical implementation Fundamental performance limitations of distributed asynchronous systems, with concurrency constraints, with non-collocated sensors, decision making and actuation nodes, with multiple feedback loops, with delay and bandwidth constraints Distributed control of and inference in the same Theories of compositionality Much better integration of logic and optimization for trade-off analysis in dynamical systems Deeper Analysis of System Models and Properties

10. Executable Models Formal Models Performance Models Cross-Linked Executable, Formal and Performance Models for ASS

11. Executablesystem models (ESM) utilize modern software engineering methodologies to develop object-oriented and component-based models, utilizing UML2 and other advanced software systems – Rapsody, etc. From these models automatic generation of executable code for all elements is possible. Embedded in these models are semantics of the operation and composition of the various components. Formal system models (FSM) are based on communicating extended finite state machines (deterministic or stochastic) (CEFSM) or on colored timed Petri nets (deterministic or stochastic) (CTPN). They are linked with the executable models via bisimulation relationships, and typically correspond to approximations of the executable models by emphasizing timing behavior of the modeled system in a timed automata sense. Performance system models (PSM) are based on various approximate dynamic system model frameworks (queuing systems, differential equations and fluid flow, difference equations, discrete event systems) together with performance metrics that can be evaluated using the models either analytically or by efficient numerical schemes. Performance models are linked to executable models via bisimulation relationships, and typically correspond to approximations of the executable models emphasizing performance and quality metrics or bounds. Performance models are also linked to Formal models via bisimulation relationships and critical event correspondence. Cross-Linked Models

12. This is already a substantial extension from current distributed software engineering practice A further extension is that we will develop a formal compositional (or component based) version of this approach. This includes development of semantics for linking components of the software and of the system, including the associated theories of components and compositionality. This, methodology and framework is in itself an important contribution to system science. It is this specific framework and underlying mathematical methodologies that we utilize to describe, model and evaluate the structure of ASS (including software structure and architecture) versus multi-criteria (multiple metrics) performance. Represents an innovative departure from current state of the art in ASS investigations that focus almost entirely on behavior (i.e. the dynamics of the algorithms implemented by the ASS). Cross-Linked Models

13. Our framework allows us to investigate the design of both structure and operation (i.e. behavior) within a well integrated framework. A significant and unique feature of our approach is that we will be able to check correctness of functionality as well as performance of the software system or its components. Furthermore and most significantly the proposed approach and framework allows the automation (to a large degree) of the validation, verificationand testing of the software system and of its dynamic operation. Cross Linked Models