1 / 33

Model-Based Black Box Testing and its Standardization Layers

MaTeLo, a model-based testing solution by ALL4TEC, improves test efficiency by automating test case generation based on formal models. Boost your testing process with MaTeLo's integrated tools for design, execution, and reporting.

senwe
Download Presentation

Model-Based Black Box Testing and its Standardization Layers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Model-Based Black Box Testingand its Standardization Layers MaTeLo THE SOLUTION TO BOOST YOUR TEST EFFICIENCY

  2. ALL4TEC Since 1998 French: 45 experts Turnover 2010: 4 M€ Expertise offer: • PROCESS IMPROVEMENT • SAFETY ENGINEERING • SYSTEM ENGINEERING • OPTIMIZED TESTING • SOFTWARE RELIABILITY EMBEDDED SYSTEMS

  3. OUR COMPETENCES OFFER MaTeLo EXAM M Elopée Safety Architect

  4. AGENDA • THE TEST PROBLEM • THE EXPECTED SOLUTION • THE MODEL-BASED TESTING SOLUTION • A PRAGMATIC INDUSTRIAL USE-CASE • A MORE STANDARDIZED TOOLS INTEGRATION

  5. THE TESTING PROBLEM • Test Resources Lack • Test Focus Deviation Testing Effort Test Needs System Complexity Test Availability Time Usage Intensity Reliability Market Expectation Reality Product Life Customer Validation Normal Usage Development Phase

  6. HOW TO SOLVE THIS ISSUE?

  7. MODEL-DRIVEN ENGINEERING Functional Requirements DESIGN TEST Implement Scenarize Design Model Usage Model MaTeLo Generate Generate Model Based Testing Code Test Cases Compile Derive Test Environment Test Scripts EXE

  8. More Productive Coverage WHAT IS MaTeLo? Formal Test Specification Product Usage Point of View MaTeLo Fully Measured Process Markov Test Logic Test Execution Compatibility Bidirectional Requirements Traceability Systematic Engineering Productivity

  9. MATELO SCREENSHOT

  10. INTEGRATED TEST WORKFLOW PLAN MaTeLo Usage Model Editor Test Design into Usage Models ACT MaTeLo Testor Automatic Test Cases Generation MaTeLo Test Campaign Analysis Confidence And Coverage Reporting EXAM Test Automation CHECK DO

  11. MaTeLo EDITORUsage Model Design

  12. MATELO USAGE MODELS • Probability on every Alternatives • Association of Requirements • Configuration of Test Operations • Setup Transition as Test Step BEGINNING TRANSITION MACRO CHAIN STATE END

  13. MODEL TRANSITION = TEST STEP Select_Gear(5) Accelerate(100%) Check_Speed(185) Requirements Test Operations Stimulations Inputs Stimulation Equivalence Classes Timing • Test Oracle • Outputs = f(Inputs) • Verifications • Expected Outputs • Timing

  14. MATELO PROFILE • Profiles can be embedded to qualify the usage model • Operational profiles • Test profiles 87 (+/- 25) Data distribution Usage path probability 347 0-100-150-200-500

  15. MaTeLo EDITORRequirements Management

  16. REQUIREMENTS MANAGEMENT IBM Rational Doors MaTeLo Requirements Library MaTeLo Usage Model Import Association (N,M) Requirement Update New Baseline Quick Model Update Update Requirements Change Analysis Available for other ReqMgt tools with import of XML or CSV files Model Impact Highlighting

  17. MaTeLo TESTORTest Cases Generation

  18. Usage approach Most probable approach Risk based approach Arcs coverage approach Start Start Start Start Custom Test profile Usage Test profile Close Close Close Close FREQUENCY FOCUS CRITICALITY, COMPLEXITY UPDATE FOCUS REQUIREMENTS COVERAGE OPERATIONAL COVERAGE MaTeLo TEST STRATEGY • DEFINE THE TEST STRATEGY, BY CHOOSING • Test Algorithm • Test Profile • Part of model to test Generate

  19. Requirement Test Function Input State ExpectedResult Verdict MaTeLo TESTOR: HTML TEST PLAN

  20. Example: MaTeLo FOR EXAM • Test Repository • SUT Interface • Test Operations • Stimulation • Measurement • Administration • Sub Test Sequences • Requirements • Name • UUID • Description Association • Automatic Test Cases • Automatic call of Test Operation • Automatic parameterisation • Requirements association • Test Case description generation • Usage model respect Association Generate

  21. MaTeLo TCATest Campaign Analysis

  22. Metrics Quality/Version New Test Cases Generation New Package Bugs New version TEST CAMPAIGN PROCESS • Target Quality Criteria • Coverage • Reliability • Confidence Indicators • Operational Confidence • Needs Coverage • Functional Coverage

  23. From Pragmatic Tools Integration To Its Standard Adoption

  24. THEORETICAL MBT WORKFLOW Automation Test Framework Atomic Test Action Confidence & Coverage Reports Test Scheduler Test Scripts Usage Model Test Cases Test Cases Verdicts Simulator Test Execution Log Functional Requirements System Under Test

  25. A 1ST INDUSTRIAL TOOLS INTEGRATION EXAM Automation Test Framework Atomic Test Action Confidence & Coverage Reports Test Scheduler UML Test Scripts Python Usage Model Test Cases Test Cases Verdicts MaTeLo dSpace MicroNova Simulator Carts Test Execution Log Functional Requirements Doors Vision CAN System Under Test IOs

  26. A 2nd INDUSTRIAL TOOLS INTEGRATION EXAM TestStand Automation Test Framework Atomic Test Action Confidence & Coverage Reports Test Scheduler UML Test Scripts Python Usage Model Test Cases Test Cases Verdicts Proprietary MaTeLo dSpace NI Opal-RT MicroNova Simulator Clemessy Carts Test Execution Log Functional Requirements Doors Reqtify Vision CAN System Under Test Diagnostic Calibration MS Excel IOs Radio

  27. VARIOUS TOOLS INTEGRATION EXAM TestStand Automation Test Framework Atomic Test Action Confidence & Coverage Reports CANoe EXACT PROVEtech Proprietary Test Scheduler UML TTCN-3 Test Scripts Python VB Usage Model Test Cases Test Cases Verdicts CSV C, C# Proprietary Java MaTeLo dSpace NI Opal-RT MicroNova Simulator Clemessy Carts Test Execution Log Simulink CANoe Functional Requirements Doors Reqtify Vision CAN System Under Test HP QC CSV Diagnostic Calibration MS Excel XML Flexray IOs Radio Temperature

  28. VARIOUS TOOLS INTEGRATION EXAM TestStand Automation Test Framework Atomic Test Action Confidence & Coverage Reports CANoe EXACT PROVEtech Proprietary Test Scheduler UML TTCN-3 Test Scripts Python VB Usage Model Test Cases Test Cases Verdicts CSV C, C# Proprietary Java MaTeLo dSpace NI Opal-RT MicroNova Simulator Clemessy Carts Test Execution Log Simulink CANoe Functional Requirements Doors Reqtify Vision CAN System Under Test HP QC CSV Diagnostic Calibration MS Excel XML Flexray IOs Radio Temperature

  29. MBT AND ITS STANDARDS EXAM TestStand Automation Test Framework Atomic Test Action Confidence & Coverage Reports CANoe EXACT PROVEtech Proprietary Test Scheduler ATX UML TTCN-3 ATX Test Scripts Python VB Usage Model Test Cases Test Cases Verdicts ATX ATX CSV C, C# Proprietary Java OTX HIL API MaTeLo dSpace NI Opal-RT MicroNova Simulator RIF ODS Clemessy Carts Test Execution Log Simulink CANoe Functional Requirements FIBEX XCP Doors Reqtify Vision CAN System Under Test HP QC CSV ODX GDI Diagnostic Calibration MS Excel XML Flexray IOs Radio Temperature

  30. QUESTION ? www.all4tec.net Sales contact anthony.faucogney@all4tec.net +33 6 80 88 40 59 Wiki & Forum & Documentation www.all4tec.net

  31. ABSTRACTION LAYERS Usage Scenario Description Markov-Chain Usage Model MaTeLo ATX << derive >> Test Case Specification UML Sequence Diagram << generate >> HIL API Test Flow Control Device Driver Python Precompiled Application ODX, FIBEX, XCP… << control >> System Under Test Hardware In the Loop Test Bench Third Party Equipment

  32. EXAM ABSTRACTION MANAGEMENT Possible Usage Scenarios Strategic Test Cases ATX Test Case 2 Test Case 1 Test Case n Available Test Configurations HILPlateform Network Functions Test Tools Env. Model HIL API ODX, FIBEX, XCP… HIL API Manual Gear Gasoline dSpace CANoe CAN_1 Auto Gear NovaSim CANape CAN_2 Diesel Hand Free Turbo LIN_X Carts INCA ASAMHIL API FlexRay_1 MS Excel Hybrid ACC Proprietary FlexRay_2 StartStop Diag Tool Electric … … … … …

More Related