110 likes | 238 Views
Using Symbolic PathFinder at NASA. Corina P ã s ã reanu Carnegie Mellon/NASA Ames. Introduction. Modern software Multi-core, cloud-based, cyber-physical Needs to be highly reliable and secure Testing Most widely used May miss important errors Formal analysis
E N D
Using Symbolic PathFinder at NASA Corina Pãsãreanu Carnegie Mellon/NASA Ames
Introduction • Modern software • Multi-core, cloud-based, cyber-physical • Needs to be highly reliable and secure • Testing • Most widely used • May miss important errors • Formal analysis • theorem proving, model checking, symbolic execution, static analysis, etc. • Strong guarantees of reliability and safety • Goal • Use formal techniques to make testing more effective
Symbolic PathFinder (SPF) • Performs • Automated test case generation • Property checking • Integrates • Symbolic execution, model checking and constraint solving • Uses analysis engine of Ames’ JPF tool Test case generation – the easiest path towards the infusion of advanced formal verification technologies JPF
Uses of SPF • Analysis/test generation for • Byte-code • Models (Simulink/Stateflow, UML) • Translation into Java • Polyglot Framework – w/ U. Vanderbilt, U. Minnesota • SPF handles • Pre-conditions, data structures, concurrency • User customizes • Coverage, search strategy, output format • Used at NASA, Fujitsu, academia Polyglot Orion orbits the moon (Image Credit: Lockheed Martin)
OAE Structure Inputs Checks Flight Rules to see if an abort must occur Select Feasible Aborts Pick Highest Ranked Abort Experience: Onboard Abort Executive (OAE) Prototype for CEV ascent abort handling being developed by JSC GN&C Results • Baseline • Manual testing: time consuming • Guided random testing could not cover all aborts • Symbolic PathFinder • Generates tests to cover all aborts and flight rules • Found major bug in new version of OAE • Environment constraints • Input data is constrained by environment/physical laws Example: inertial velocity can not be 24000 ft/s when the geodetic altitude is 0 ft • Need to encode these environment constraints explicitly • Solution • Use simulation runs and machine learning to get data correlations
Lessons Learned • Not enough to generate test cases that achieve good coverage • Should also make sense to developers • Environment assumptions need to be made explicit • Using both input from the user and automated inference techniques • True for any analysis tool that can only analyze components (not entire systems)
Usability Challenges • Tool Usability • Research Challenges • Domain Specific
Tool Usability • Easy set-up -- Not true for JPF/SPF • Portability for multiple platforms • Animation and/or graphical display of results • Intermediate results • Important for tools that do not give “instant” results like SPF • Informative counterexamples/Explanation of errors • Documentation easily accessible on-line -- JPF’s wiki • Integration with other tools familiar with developers • JPF’s Eclipse plug-in; SPF outputs JUnit tests and Antares scripts • Clear display of different options --Not true for JPF/SPF • Availability --JPF is open-source • …
Research Challenges • Scalable and timely verification • Analysis as efficient as possible • Parallelization, compositional techniques, fast constraint solvers • New constraint solvers • Solve complex, non-linear constraints (NASA applications) • Solve strings combined with numeric constraints (web applications, security) • Automatic discovery of interfaces • With (minimal) inputs from user • Combinations of automata and machine learning techniques • …
Domain Specific Knowledge • Add the user to the picture • Integration of domain specific knowledge • Adaptation of tools to users’; specific needs • E.g. NASA uses models; adapt SPF from code to models; customize search strategies/output format • …