1 / 42

From Validating Models to Validating Systems

From Validating Models to Validating Systems. Peter Denno 2013-02-25 University of Maryland ISR Colloquium. Outline. Introduction / Scoping Requirements for MBSE Exchange Form Validation NIST Work. Goals. Describe a “design philosophy” for systems that assist in systems engineering

iram
Download Presentation

From Validating Models to Validating Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. From Validating Models to Validating Systems Peter Denno 2013-02-25 University of Maryland ISR Colloquium

  2. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work

  3. Goals • Describe a “design philosophy” for systems that assist in systems engineering • Framework for linking multiple viewpoints • Framework for research • Link the design philosophy to NIST workin exchange form validation, requirements engineering, supply chain logistics simulation

  4. What is special about V&V ? (1) • IBM Watson • New techniques – exceed human capability in knowledge-intensive tasks • “Machine understanding is not human understanding.” • “Knowledge is not the destination.”

  5. What is special about V&V ? (2) • Validation & Verification • Knowledge is the destination. – knowledge, or at least credible rationale. • Requirement: • Minimally: be able to explain how the design space was characterized and demonstrate that requirements are being met. • Ideally: Provide deductive arguments where appropriate • Show how certain alternatives are indeed incompatible • Reference principles of operation, functions

  6. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work

  7. Basis for SE decision making • SE decision making macro level: • Trade studies, simulations, risk assessment, etc. • SE decision making micro level: • A web (conceptual schema) of information • Uncertain • Conflicting • Isolated • Uncertainty is quantified • Conflicts resolved • Inter-relation revealed

  8. Strategy for the micro-level information • Characterize elements of rationale for SE decision making. • Each research project touches on only a few of these elements • No single overarching system design intended

  9. 9 Elements of Rationale (1) • Measurement Conditions • Confidence in the process or environment under which it was measured • “Capacitance was measured using the AC impedance technique.” • Logical Consistency • Confidence due to consistency with theory. Type consistent. • “P = .05, as we’d expect from the law on conservation of energy.” • Associativity Across Views • Individuals: knowledge that two references made from different viewpoints refer to the same thing • “The region P on the CAD model corresponds to these elements in the FEA mesh.” (Individuals) • Concepts: knowledge that two conceptualizations can be used for the same purpose. • “What the supplier is calling ‘rated maximum pressure’ is what we call ‘rated pressure.” (Concepts)

  10. 9 Elements of Rationale (2) • Change process • Knowledge of precursors and the history of properties that distinguish them. • “The value of P that we calculated for this design is close to what we found in earlier models.” • Authority • The power that information has due to an approval that is granted or an estimate of its maturity • “Supplier-provided data also suggest P=.05 is obtainable.” • Origin in Requirements* • Belief that a requirement is sensitive to it • “Our ability to achieve requirement x diminishes as P exceeds 0.07.”

  11. 9 Elements of Rationale (3) • Origin in organization infrastructure • Belief because you obtained it in ways consistent with the organization’s best practices. • “P was obtained from the aero model in the preliminary design library.” • Consistency with other belief • Belief due to consistency with prevailing contingent facts • “P=0.5 is reasonable in products using component y.” • V&V Process • Belief that the system in place to manage the other 8 elements is sound and comprehensive. • “The value of P is confirmed through simulation that is routinely performed in validation of this product line.”

  12. 9 Elements : Observations • Coupling and overlap • Authority / Origin in Organizational Infrastructure • Associativity across views / measurement conditions • etc. • Though these are found in models, they can be expressed from a more comprehensive viewpoint where • Contradictions can be exposed • Cohesion across views can be noted • Trace to requirements is more evident • (These are all parts of V&V)

  13. MBSE Concepts / Logical View

  14. Sentence detail / rationale

  15. Example Usage Patterns • V & V • Origin in requirements • Automated generation of test cases • Requirements Engineering • Origin in other belief, emphasis on tracking contingent facts and engineering change • Refinement

  16. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work

  17. Exchange Form Validation : Two Methods • Axiomatic: • How: Map the exchanged content to sentences • Identify errors: ex falsoquodlibet with a reasoner • Advantage: Ontology explains intent • Disadvantage: Proofs hard to interpret • Metamodel: • How: Map the exchanged content to objects • Identify errors: Direct structural, with OCL, etc. • Advantage: Constraints relate to exchange form • Disadvantage: Constraints look like code

  18. Example use of metamodel View / Viewpoint: Can be both consistent with a form (a view), and the form by which otherconceptualization are stated (a viewpoint.)

  19. Example from the UML Metamodel

  20. Example Specification Constraints

  21. In MBSE, metamodels play a key role • Metamodel = (1) a specification of the form a model can take. (well-formedness conditions) (2) a formalization of the viewpoint that models will express

  22. Metamodels also play a key role in model exchange • Metamodel = (1) a specification of the form a model can take. (well-formedness conditions) Definition of structure  serialization (2) a formalization of the viewpoint that models will express Illuminate what program structures the elementsof exchange content map to/from.

  23. Communication with Exchange Standards

  24. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work • Model Interchange Working Group • Supply Chain Logistics Simulation • Collaborative Requirements Engineering

  25. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work • Model Interchange Working Group • Supply Chain Logistics Simulation • Collaborative Requirements Engineering

  26. OMG Model Interchange Working Group • Goal: Improve the ability of OMG MOF-based tools (UML, SysML) to exchange information • XMI Serialization – common to MOF-based tools. • Process • Group: Produce Test Case diagram and reference file • Tool developers: create diagram in their tool, serialize as XMI • Use NIST tool to identify errors (in files and metamodels) • Correct tools and specifications

  27. NIST UML / SysML Validator Enter below a file to upload:

  28. NIST UML / SysML Validator

  29. NIST UML / SysML Validator

  30. NIST UML / SysML Validator

  31. MIWG Results • Stakeholders witness significant improvement in interoperability Elaasar & Labiche, 2012

  32. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work • Model Interchange Working Group • Supply Chain Logistics Simulation • Collaborative Requirements Engineering

  33. Supply Chain Logistics Simulation • Goal: Demonstrate integrated use of models toward enterprise goals • Design: Map models, guided by metamodels into sentences that guide compilation of a discrete event simulation. • Models • UML of ordering / logistics objects • QVT-r mapping of messages to orders • BPMN “stereotyped” + OCL of business decisions • Discrete Event Simulation

  34. Round Trip Engineering of Supply Chain Logistics

  35. Logistics Processes (1)

  36. Logistics Processes (2)

  37. Business Rule

  38. Simulation Results

  39. Outline • Introduction / Scoping • Requirements for MBSE • Exchange Form Validation • NIST Work • Model Interchange Working Group • Supply Chain Logistics Simulation • Collaborative Requirements Engineering

  40. Collaborative Requirements Engineering • Goal: Demonstrate Engineering from Product Data Sheets • Design: Map product data sheets in to sentences about requirements. Use these to guide engineering simulation and reasoning about alternative designs

  41. Conclusions • Continuing roles for deductive reasoning in the automation of SE processes • The nature of V&V, requirements engineering, the way we think when we engineer, require it. • Preparing and interpreting macro-level SE decision processes is aided by the integration of multi-viewpoint, micro-level information. • Metamodels facilitate this integration.

  42. References • Welty, C; Inside the mind of Watson, 2nd ESWC Summer School, Kalamaki, 2012, http://videolectures.net/eswc2012_welty_watson • Denno, P; Thurman, T, Mettenburg, J; Hardy, D; On enabling a model-based systems engineering discipline – 18th INCOSE International Symposium (2008) • Denno, P; Harrison, T; Using Legacy Modeling Artifacts in Supply Chain Logistics Simulation (in draft, 2013) • ISO 15288 (2008) – Systems and software engineering – System life cycle processes. (2008)

More Related