Scenario Based Testing of Advanced Driver Assistance Systems

1. Scenario Based Testing of Advanced Driver Assistance Systems Date: 2010 Time:

2. Speaker Introduction MS Kausalya Paila kausalya.p@embitel.com • Masters in Power Electronics and Bachelors in Electrical & Electronics • Working in Software development and verification for the last 7years • Has several publications on model based development and testing

3. Agenda • Introduction • Terminology • Advanced Driver Assistance Systems (ADA systems) • Sensor Fusion • Model Based Development • Model Based Testing (MBT) • Scenario Based Testing • Need for Scenario Based Testing in MBT • Prototype tool developed for SBT • Case study & Results • Summary • Q&A

4. Advanced Driver Assistance System • Helps driver to avoid or mitigate an accident • Depending on the significance and timing of the threat, these systems will: • ALERT the driver to an impending danger • WARN if there is no driver reaction and • ASSIST or ultimately INTERVENE • Existing ADA systems are Lane Departure Warning, Collision Preparation System and Forward Collision Warning

5. Sensor Fusion Algorithm • Sensor fusion is the combining of data from several sensors • Fusion improves the accuracy of data by giving out a union of data from these sensors • Examples of inaccuracy of data are • Width of an object detected is different from different sensors • Radar data may not be accurate in bad weather conditions • No of objects detected may vary based on positioning of sensors Radar Laser GPS Inertial Measurement Video Fusion ADA System Union of Sensor Data Alert

6. Traditional Design vs. MBD • Traditional Design Life Cycle • Design information in text form – difficult to comprehend • Manual coding – more error prone and time consuming • Little Tracking to Change Requests • Discovery of design errors late in design cycle – Delays • Model Based Development • Single design environment • Developers can evaluate multiple options, predict system performance • Possibility of accurate production code generation

7. Model Based Development - Detail • - Plant Models • - Control System Design • - Rapid-Prototyping • - SIL • System Simulation Validation in Vehicle Define System & Controller Requirements HIL Generated code Documentation Controller Validation & Calibration Concept Design -Test vector generation -Functional tests -SIL -Test Cases from reqs -Algorithm Models -Rapid-Prototyping -MIL & SIL -System Simulation Unit & Subsystem Testing Algorithm Design -Algorithm Models -Modeling Guidelines -Auto code generation Implementation/ Auto Code

8. Model Based Testing • Largest possible share of the testing is done at the model level • Executable models as a main source of information for testing • Test information from the model stage can be reused for subsequent testing (SIL&HIL) • Test scenario creation from the functional specification and requirements-based testing forms the focal point of the dynamic testing workflow • Model test vectors to be derived from the software requirements specification according to established criteria

9. Scenario Based Testing • A scenario is used to help a person think through a complex problem or system • Ideal scenario test has characteristics…. • A scenario test is based on a story or a sequence of events about how the application is used, including information about the motivations of the other systems or people around • The story involves a complex use of the application or a complex environment or a complex set of data. • The test results are easy to evaluate.

10. Why Scenario Based Testing • Learn the product early in testing • Connect testing to requirements specifications • Expose failures to deliver desired benefits • Expert use of the application • Pop-up requirements-related issues, which might involve reanalyzing already defined requirements • Surfacing not-yet-defined requirements

11. Why SBT in DA Systems • An Example Requirement: • Six Objects of Interest (OOI) shall be passed from the Object Selection function to the AACC function. In all cases the object may be stationary, moving in the same direction or moving in the opposite direction relative to the host vehicle. • OOI[2] is the nearest OOI in the adjacent lane to the left. • Test Vector generation for such requirements is not only difficult with existing COTS tools but also is difficult to visualize from a tester’s perspective • Assertion based testing could be a solution but tester’s effort is very large in creating assertions for all possible scenarios • Scenario based testing helps in the cases similar to above where user can create a scenario and generate test vectors

12. A Scenario Based Testing Tool • A prototype is done as proof of concept • Physical system behavior is modeled • Least Square Estimation technique is used for estimation algorithm (LSE will be replaced by Kalman filter technique in next version) • Vehicle parameters and environmental parameters shall be configured by the user in order to run the simulation • Test vectors are generated for the simulation duration specified by user • Test vectors can be saved by the user in excel or mat format • Future plan is to make the tool configurable

13. A Case Study • The prototype tool is configured to test Fusion Application • Motivation behind Choosing Fusion Algorithm: • Complexity • Testing such an application is laborious because of the large no of inputs and outputs handled by the application • Accurate testing can be done with defined scenarios • Need to use a tool which can combine both the automated test vector generation process and scenario based testing approach

14. Example…

15. Example…

16. Results

17. Summary • Terminology – Driver Assistance Systems, Fusion • MBD • MBT • SBT • Need for SBT in MBD • A new concept or tool to realize SBT • Tool details • Results

18. Thank you for your interest! Any Questions? Ms Kausalya Paila kausalya.p@embitel.com http://in.linkedin.com/in/kausalyap Embitel Technologies(India) Pvt Ltd. e-mail: sales@embitel.com www.embitel.com www.linkedin.com/companies/embitel