Which role might model-based engineering play in software certification?

1. Which role might model-based engineering play in software certification? Selo Sulistyo

2. Outline • Software Certification • Model-based Engineering (MBE) • Roles of MBE in Software Certification • Summary

3. Software Certification Definition Software certification is the term used to certify the quality of a software application/product against the functional and non-functional specifications before its release to production/ end-user. (Eushiuan Tran, 1999)

4. Software Certification Why is a certification needed? The software certification process establishes the level of confidence in a software system in the context of its functional and safety requirements.

5. Development process Programming Languages Is compiled using Source Code Compiler Generates Generates Assembly byteCode Assembler is executed by Virtual Machine Running Application Machine Code is executed and managed by is executed and managed by is executed and managed by Software Certification What is being certified? Is used to produce application Product Operating System Computer Hardware

6. Software Certification Certification Approaches • Product based Example: ISO IEC 14598. This international standard provides guidance for the practical implementation of software product evaluation when several parties need to understand, accept and trust evaluation results. • Process-based Example: IEC 61508(Functional safety of electrical/electronic/ programmable electronic safety-related systems) • DO-178B(Software Considerations in Airborne Systems and • Equipment Certification).

7. Software Certification Certification process Verification and validation are part of the certification process for software system (Eushiuan Tran, 1999) Verification Validation Certification Developer Functional Expert Certification Authority

8. iteration Deve-lopment Was the right product built? Was the product built right? Testing Development Testing Software Certification When and how the verification and validation are done? requirements artifacts Requirement analysis design implementation verification deployment Validation

9. Software Certification Verification The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. Was the product built right? • Verification does not ensure the application: • – Solves an important problem • – Correctly reflects the workings of a real world process

10. Software Certification Validation The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. Was the right product built?

11. Software Certification Verification and Validation Techniques

12. Software Certification An example : DO-178B

13. Software Certification An example : DO-178B

14. Outline • Software Certification • Model-based Engineering (MBE) • Role of MBE in Software Certification • Summary

15. requirements Requirement analysis Models design implementation verification deployment Validation Model-based Engineering Definition • An approach to engineering that uses models as an integral part of the technical baseline that includes • the requirements, • analysis, • design, • implementation, and • verification of a capability, system, and/or product throughout the acquisition life cycle.

16. Model-based Engineering What is a model? • A physical, mathematical, or otherwise logical representation of a system, entity phenomenon or process • Models are scoped to purpose • Models are appropriate to the context (e.g., application domain, life cycle phase) • Models integrated or interoperable across domains and across lifecycle

17. Modeling language is written in describes model system Model-based Engineering Software Model

18. requirements Requirement analysis Deve-lopment design implementation verification Testing deployment Validation Model-based Engineering The use of model in software development • Jos B. Warmer and Anneke Kleppe defined Modeling Maturity levels (MML) • MML 0: No Specification • MML 1: Textual Specification • MML 2: Text with Models • MML 3: Models with Text • MML 4: Precise Models • MML 5: Models only

19. requirements My software … Requirement analysis Deve-lopment design implementation verification Testing deployment Validation Model-based Engineering MML 0 No Specification No specification

20. requirements Requirement analysis Deve-lopment design implementation verification Testing deployment Validation Model-based Engineering MML 1 Textual Specification • Specification of software is written down in one or more natural language documents.

21. requirements Requirement analysis Deve-lopment design implementation verification Testing deployment Validation Model-based Engineering MML 2 Text with Models • Specification of software in one or more natural language documents plus several high-level diagrams to explain the overall architecture. +

22. requirements Requirement analysis Deve-lopment + design implementation verification Testing deployment Validation Model-based Engineering MML 3 Models with Text • Specification of software is written down in one or more models. Additional natural language text is used to explain the background and motivation of the models.

23. requirements Requirement analysis Deve-lopment design implementation verification Testing deployment Validation Model-based Engineering MML 4 Precise Models • Specification of software is written down in one or more models. • Natural language text is used to explain the background and motivation of the models. • The models are precise enough to have a direct link with the actual code.

24. requirements Requirement analysis Development design implementation verification Testing deployment Validation Model-based Engineering MML 5 Models only • The models are precise and detailed enough to allow complete code-generation. • The code is invisible (as assembler is today). • Modeling language  High level programming language.

25. MBE ! Model-based Engineering Modeling Maturity Levels (MML) • MML 0: No Specification • MML 1: Textual Specification • MML 2: Text with Models • MML 3: Models with Text • MML 4: Precise Models • MML 5: Models only

26. Model-based Engineering OMG’s Model-driven Architecture • The current state of the art in MDE is much influenced by the ongoing standardisation activities around the OMG Model Driven Architecture® (MDA®). • MDA is a framework which defines a model-driven approach to software systems development. • Focus on the solution of the problem rather than the implementation problem • Raising the level of abstraction, dealing with complexity • An automatic code generation

27. PSM PSM PSM CIM PIM Model-based Engineering Models in MDA (Model-Driven Architecture) RequirementAnalysis CIM: Computational-independent models PIM: Platform-independent Models PSM: Platform-specific Models TT High-level design TT TT TT TT: Transformation Tool Low-level design TT TT TT Code Code Code Implementation

28. Outline • Software Certification • Model-based Engineering (MBE) • Role of MBE in Software Certification • Summary

29. Models Transformation tool e.g Code Generator Running Application Role of MBE in Software Certification How does model-based engineering produce applications? Modeling Language Tools and Production processes ?? Products

30. Role of MBE in Software Certification Certification • One central goal of MBE is to enable analysis of the system, thus ensuring the quality of the system already on the model level. •  early verification and validation

31. How to verify these software models? Model Verification Models Model Validation Was the model built right? Verified and Validated Software Models verification deployment Validation Role of MBE in Software Certification When and how the verification and validation are done? requirements Was the right model built? Requirement analysis Was the model built right? design implementation Certification Authority …..

32. Role of MBE in Software Certification Model verification • Verification is done to ensure that: • – The model is developed correctly and conformance with standard • – The algorithms have been implemented properly • – The model does not contain errors or bugs • Verification does not ensure that the model: • – Solves an important problem • – Meets a specified set of model requirements • – Correctly reflects the workings of a real world process

33. Role of MBE in Software Certification Verification techniques Verification techniques that can be used: • – Model checking • – proof of correctness • – consistency checking • This will ensure that the models are • Consistent • Correct against to language syntax of the language

34. Role of MBE in Software Certification Model validation • Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained.  Compliance with requirements • The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to make the model actually used.

35. A TT B A TT B B A A TT PSM PSM PIM CIM B B Code Role of MBE in Software Certification Model verification and validation • B: Validation • Compliance with requirements • Traceability • A: Verification • Consistency • Conformance with standard TT: Transformation Tool TT TT Code Product family Product family

36. Role of MBE in Software Certification Benefits MBE in software certification • Models provides a view to a complex problem and its solutions. • Models are less risky, cheaper to develop and easier to understand than the implementation of a genuine target system • By assuming that the transformation tools is verified then the verification process can be done in model level, that obviously it saves time and cost. • It also improves the quality of design and software code by better assuring the traceability between artifacts.

37. Summary MBE can play role on: • Process-based certification. • Early verification process. Activities for verification, validation, and test with Model-Based Engineering can be applied at every stage of the development process. • The model traceability will minimize the certification efforts, especially for product families.  reducing certification and re-certification times, • Saving time and cost since the verification is done on higher abstraction levels, but it may be a complicated process with the current MDE technology.

38. Thank You

39. References Jos Warmer, The role of OCL in the MDA, Klasse Objecten, http://www.klasse.nl Eushiuan Tran, Verification/Validation/Certification, Carnegie Mellon University, 18-849b Dependable Embedded Systems, Spring 1999 Rajwinder Kaur Panesar-Walawege, Mehrdad Sabetzadeh, Lionel Briand, Using Model-Driven Engineering for Managing Safety Evidence: Challenges, Vision and Experience, Simula Research Laboratory, University of Oslo, Lysaker, Norway Ewen Denney and Bernd Fischer, Software Certification and Software Certificate Management Systems, USRA/RIACS, NASA Ames Research Center, Moffett Field, CA 94035, USA OSMAN BALCI, A Methodology for Certification of Modeling and Simulation Applications, ACM Transactions on Modeling and Computer Simulation, Vol. 11, No. 4, October 2001, Pages 352–377. Damian Dechev, Bjarne Stroustrup, Model-based Product-Oriented Certification, Texas A&M University College Station, TX 77843-3112