Jinlin Yang and David Evans [jinlin, evans]@cs.virginia Department of Computer Science

1. Dynamically Inferring Temporal Properties Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science University of Virginia PASTE 2004 June 7th 2004 http://www.cs.virginia.edu/ipa

2. Temporal Properties • Example: fopen/fclose • Essential for program’s correctness • Hard for humans to document correctly [Holzmann, FSE ’02 keynote] • Can we infer them automatically? Yes! Jinlin Yang

3. Related Work • Value-based invariants • Daikon [Ernst, TSE, Feb. ‘01] • We focus on invariants in sequence of events • Temporal properties • Recovering thread models [Cook, JSS ‘04] • Mining specification [Ammons, POPL ‘02] • Extracting component interfaces [Whaley, ISSTA ‘02] • We focus on relationship between 2 or 3 events: greater automation, scalability Jinlin Yang

4. Our Approach Program Inferred Properties Instrumented Program Execution Traces Inference Instrumentation Testing Test Suite Candidate Property Patterns Jinlin Yang

5. Program Verification Derived Properties Satisfaction or Counter- examples Program Inferred Properties Validation Report Generation Dynamic Inference Report Model Jinlin Yang

6. Program Evolution Shared Properties Program Version 1 Inferred Properties 1 Dynamic Inference Difference Analyzer Program Version 2 Inferred Properties 2 Different Properties Jinlin Yang

7. Property Patterns • Response pattern [Dwyer, ICSE ‘99] [-P]* (P [-S]* S [-P]*)* SPPSPS  SPSP  • Too weak • 3 primitive patterns • 4 derived patterns Jinlin Yang

8. Partial Order of Patterns Alternating PSPS MultiEffect PSS MultiCause PPS EffectFirst SPS Stricter OneCause SPSS CauseFirst PPSS OneEffect SPPS Response SPPSPS Jinlin Yang

9. Logical Relation of Patterns Alternating PSPS • For each combination of two events • Decide if they satisfy CauseFirst, OneCause, or OneEffect • Find the strictest pattern Λ MultiEffect PSS MultiCause PPS EffectFirst SPS Stricter Λ Λ Λ OneCause SPSS CauseFirst PPSS OneEffect SPPS Response SPPSPS Jinlin Yang

10. Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces Jinlin Yang

11. Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces Jinlin Yang

12. Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces CauseFirstΛOneEffect  MultiCause Jinlin Yang

13. Implementation Program Inferred Properties Instrumented Program Execution Traces Instrumentation Testing Inference • Automated testing and inference • Instrumentation and properties diff by hand • 900 lines of Perl code • 93 different events, 100 traces, each has 222 events on average: less than an hour Test Suite Candidate Property Patterns Jinlin Yang

14. Experiments • Hypotheses • We can automatically extract interesting temporal properties. • Differences of temporal properties among multiple versions can reveal interesting things. • Target programs • Tour bus simulator (8 student submissions) • OpenSSL (0.9.6, 0.9.7-0.9.7d) • Simplified producer-consumer (in the paper) Jinlin Yang

15. Tour Bus Simulator • Bus and each passenger is a separate thread • Assignment in Fall 2003 graduate-level course (before we started this project) • 8 submissions from the instructor Jinlin Yang

16. Testing N, the number of people C, the capacity of the bus T, the number of trips Executed each submission with 100 randomly generated inputs, where 20 < C ≤ 40 C+1 ≤ N ≤ 2C 1 ≤ T ≤ 10 >cville_bus –N 2 –C 1 –T 2 Bus waiting for trip 1 Passenger 0 gets in Bus drives around Charlottesville Passenger 0 gets off Bus waiting for trip 2 Passenger 1 gets in Bus drives around Charlottesville Passenger 1 gets off Bus stops for the day Jinlin Yang

17. Results: Tour Bus Simulator • Differences reveal problems • Bus drives around before all passengers get off • Other problems Jinlin Yang

18. OpenSSL • Widely used implementation of the Secure Socket Layer protocol • 6 versions [0.9.6, 0.9.7, 0.9.7a-d] • We focus on the handshake protocol Jinlin Yang

19. Jinlin Yang

20. Testing • Manually instrumented server • Modified client: advance to a different state with 5% probability • Executed each version of server with 1000 randomly generated clients Jinlin Yang

21. Inferred Alternating Patterns Documented improvement Race condition Fixed bug 7 alternating patterns same for all versions Jinlin Yang

22. Partitioning Traces • Correct clients • Properties are exactly same as specified • All versions agree • Faulty clients without server errors generated • Found one possible vulnerability of DOS attack • Segmentation faults • A documented bug fixed since 0.9.7d Jinlin Yang

23. Summary of Experiments • Useful in program evolution • Reveal interesting changes in OpenSSL • Identify unexpected differences • Revealed bugs • Tour bus: identified faulty implementation • Multiple implementations are rare, but multiple representations are common (i.e. design, model, code) • OpenSSL: partitioning traces Jinlin Yang

24. Future Work/Research Questions • Automating instrumentation/validation • Can we automatically identify interesting events? • Identifying more useful patterns • Combine data-flow with events • More expressive, more events • Ed will talk more about this • Prioritize results • How to identify interesting differences? • Testing • How do the inferred properties vary with the test suite? Jinlin Yang

25. Conclusion • Automatically inferring temporal properties is feasible • Even very simple property patterns reveal interesting program properties Jinlin Yang

26. Thank You! http://www.cs.virginia.edu/ipa This work is funded in part by the National Science Foundation. We thank Marty Humphrey for providing the student submissions, Chengdu Huang for helping OpenSSL, Joel Winstead for discussion and comments on earlier version of this paper, anonymous PASTE reviewers for insightful comments, anonymous CS650 students who wrote the buggy code, and our research group members and CS graduate students at the UVa for helping improve the talk. Jinlin Yang