Lifecycle Verification of the NASA Ames K9 Rover Executive

1. Lifecycle Verification of the NASA Ames K9 Rover Executive Dimitra Giannakopoulou Mike Lowry Corina Păsăreanu (pcorina@email.arc.nasa.gov) Rich Washington Collaborators: Howard Cannon, Ray Garcia – ARA Group Summer Students: Colin Blundell (University of Pennsylvania), Jamie Cobleigh (University of Massachusetts)

2. cost of detecting / fixing bugs increases Requirements & Design Coding Deployment integration issues handled early Model checking/ Testing Properties, assumptions Compositional Verification Objectives Motivation / NASA Relevance: • Verification is essential for autonomy insertion into missions • Traditional testing of autonomy software is hard due to high complexity and unpredictable environments • Integration problems are difficult to detect and expensive to fix; verification should be addressed early in the software lifecycle Objectives: • Use a combination of formal analysis techniques (model checking) and testing to analyze autonomous systems throughout their lifecycle • Provide support for compositional (“divide and conquer”) verification to address scalability • Use design level artifacts to guide the implementation and to enable efficient source code verification

3. Accomplishments/Contributions • Development of novel, fully automated compositional verification techniques • Integrated lifecycle approach to verification of autonomy software Analysis of K9 Rover Executive (35KLOC of C++) • Design level modeling • Design models: describe concurrent architecture and autonomy features • Requirements: concurrency and plan execution properties • Design level analysis • Model checking used for exhaustive verification of design models • 10x improvement over monolithic (non-compositional) verification • Code level analysis • Model checking: 3x improvement over monolithic verification • Advanced testing (automated plan input generation and run time verification) • Several integration problems discovered • Change in design as a result of our analysis • Simplified architecture with increased modularity

4. Outline • Introduction • The K9 Rover Executive • Compositional Techniques for Design- and Code- Level Verification • Analysis of the Rover Executive • Conclusions and Future Work

5. Ames K9 Rover Executive • Executes flexible plans for autonomy • branching on state / temporal conditions • Multi-threaded system • communication through shared variables • synchronization through mutexes and condition variables • Architecture changed as a result of our analysis • Simplified inter-thread communication through event queue Updated architecture Original architecture

6. Outline • Introduction • The K9 Rover Executive • Compositional Techniques for Design- and Code- Level Verification • Analysis of the Rover Executive • Conclusions and Future Work

7. OK Finite-state model Verification tool or Error trace (F W) Line 5: … Line 12: … Line 15:… Line 21:… Line 25:… Line 27:… … Line 41:… Line 47:… Specification Model Checking for Design Level Analysis

8. A • Assume-guarantee reasoning: 1. check P on M1 with assumptionA for M2 2. check that M2 satisfies A • Assumption generation is a manual process P holds in system Our work: the first incremental and automated approach for assume-guarantee reasoning Solution 1 • (2002) algorithmic generation of assumption as a controller • knowledge of environment is not required • applied to Rover Executive (ASE’02) • (2003) extended to deal with deadlock • applied to DS-1 remote agent executive (Journal of ASE, 2003) Solution 2 (2003) • incremental assumption computation based on learning andknowledge of environment • applied to Rover Executive (TACAS’03, Journal on STTT, submitted October 2003) • extended framework for symmetric assume-guarantee rules (SAVCBS’03) Compositional Verification for Increased Scalability Does system made up of M1 and M2 satisfy property P? • Check P on entire system: too many states • Check one component at a time: need abstractionof other modules M1 M2 finite state models safety properties

9. counterexample – refine Ai Model Checking false M1 satisfies P, under Ai Ai Learning true P holds true M2 satisfies Ai false real error? N Y counterexample – refine Ai Learning Framework for Automated Compositional Verification • Process is iterative • We use a learning algorithm to infer the smallest assumption at each stage • Assumptions are generated by querying the system, and are gradually refined • Queries are answered by model checking • Refinement is based on counterexamples obtained by model checking • Termination is guaranteed • Algorithm is “any-time” TECHNOLOGY Extended LTSA verification tool to support automated assume guarantee reasoning at design level P violated

10. Design models P M1 A M2 P C1 A C2 PROBLEM • Suppose we have a (finite) design model for a piece of software and safety properties • Can use our automated assume-guarantee reasoning on the design model to verify the safety properties • Can we use this knowledge to improve the performance of verifying the properties on the actual implementation? TECHNOLOGY Component implementations • Developed methodology for using design-level assumptions for reasoning about source code, to improve scalability For source code verification, we used • Model checking Extended Java PathFinder model checker to support assume-guarantee reasoning at the source code level; applied approach to K9 Rover (ICSE’04) • Advanced testing Automated plan input generation based on plan language grammar Property monitoring (Eagle) SOLUTION • Use assumption A generated at the design level to check the implementation C1 || C2 in an assume-guarantee style Assume-guarantee Verification of Code with Design-Level Assumptions

11. Testing Framework Input plan & property generation Input plans Behavioural properties K9 executive instrumentation event stream Observer (Eagle) reports instrumentation

12. Outline • Introduction • The K9 Rover Executive • Compositional Techniques for Design- and Code- Level Verification • Analysis of the Rover Executive • Conclusions and Future Work

13. Analysis of the Executive Properties • Concurrency: deadlocks, correct lock usage / data races, sequence of interactions between modules • Plan execution properties Analyzed several versions • Original architecture: components (threads), locks, condition variables • New architecture: queuing and event handling, floating branch execution (execution of synchronous and asynchronous alternate plans) Design level analysis • Several synchronization issues discovered (also present in actual implementation) • 10x improvement over monolithic (non-compositional) verification Code level analysis • Model checking: 3x improvement over monolithic verification • Testing • Automatic generation of hundreds of plans in a few seconds • Detection of integration problems early (during unit testing) • Better coverage (predict, by mathematical inference, properties about all possible inter-leavings)

14. Analyzed Properties • P1: If the last task in the plan terminates successfully, then the only possible outcome for the plan is successful termination. • P2: When a task fails, the continue-on-failure flag on the block will always be checked before any outcome is produced; moreover, if continue-on-failure is true, the outcome is success, otherwise it is failed. • P3: The Executive only receives ExecCondChecker events if it has registered for them. • P4: The ExecCondChecker only puts events in the queue if the Executive registered for them. • P5: When a task fails, it will always check its continue-on-failure flag; moreover, if the continue-on-failure flag is false, no subsequent task in the block will be started; new tasks can be started after the parent block reports the results (i.e. other block is expanded). • P6: If a task fails, then the parent block’s continue-on-failure flag will be checked: if it is true, then the block succeeds, otherwise it fails. • P7: If the Executive thread reads the value of the shared variable savedWakeupStruct, then the ExexcCondChecker thread should not read it until the Executive clears it first. • P8: (Race condition) All accesses to shared structure conditionSetChanged by the Executive and the ExecCondChecker threads will be protected by locks. • P9: (Race condition) All accesses to shared structure existConditions by the Executive and the ExecCondChecker threads will be protected by locks. • P10: Absence of local and global deadlocks. • P11: No irrelevant action execution events can happen. • P12: No irrelevant condition checker events can happen. • P13: Alternate and principal plans

15. Example property (P7) plan ExecTimerChecker Executive Internal savedWakeupStruct ActionExecution Database (state of system) DBMonitor ExecCondChecker If the Executivethread reads the value of variable savedWakeupStruct, the ExecCondChecker thread should not read this value unless the Executive clears it first.

16. Example Property (P7) executive.savedWakeupStruct.read[0..1] 0 1 execCondCh.savedWakeupStruct.read[0..1] executive.savedWakeupStruct.assign[0] execCondCh.savedWakeupStruct.assign[0..1] error

17. Generated Assumption Assumption = Q0, Q0 = ( executive.exec.lock -> Q2), Q2 = (executive.exec.unlock -> Q0 | executive.savedWakeupStruct.read[1] -> Q3 | executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5), Q3 = ( executive.savedWakeupStruct.read[1] -> Q3 | executive.savedWakeupStruct.assign[0] -> Q4), Q4= ( executive.exec.unlock -> Q0 | executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5), Q5 = ( executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5). not displaying sink state Q1

18. Analysis Results for P7

19. Conclusions and Future Work • Lifecycle verification and validation of K9 executive • Modeled and checked key autonomy features at design level • Found error in design (before coding); error fixed by developer • Influenced developer to improve/simplify design • Developed testing framework for code verification • Tool for automated input plan generation, property monitoring • Developed novel compositional verification techniques for increased scalability • Future plans • Verification and validation for future universal executive (PLEXIL) • Automated verification of plans generated by Ames planner