260 likes | 366 Views
Dynamically Inferring Temporal Properties. Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science University of Virginia PASTE 2004 June 7 th 2004 http://www.cs.virginia.edu/ipa. Temporal Properties. Example: fopen/fclose
E N D
Dynamically Inferring Temporal Properties Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science University of Virginia PASTE 2004 June 7th 2004 http://www.cs.virginia.edu/ipa
Temporal Properties • Example: fopen/fclose • Essential for program’s correctness • Hard for humans to document correctly [Holzmann, FSE ’02 keynote] • Can we infer them automatically? Yes! Jinlin Yang
Related Work • Value-based invariants • Daikon [Ernst, TSE, Feb. ‘01] • We focus on invariants in sequence of events • Temporal properties • Recovering thread models [Cook, JSS ‘04] • Mining specification [Ammons, POPL ‘02] • Extracting component interfaces [Whaley, ISSTA ‘02] • We focus on relationship between 2 or 3 events: greater automation, scalability Jinlin Yang
Our Approach Program Inferred Properties Instrumented Program Execution Traces Inference Instrumentation Testing Test Suite Candidate Property Patterns Jinlin Yang
Program Verification Derived Properties Satisfaction or Counter- examples Program Inferred Properties Validation Report Generation Dynamic Inference Report Model Jinlin Yang
Program Evolution Shared Properties Program Version 1 Inferred Properties 1 Dynamic Inference Difference Analyzer Program Version 2 Inferred Properties 2 Different Properties Jinlin Yang
Property Patterns • Response pattern [Dwyer, ICSE ‘99] [-P]* (P [-S]* S [-P]*)* SPPSPS SPSP • Too weak • 3 primitive patterns • 4 derived patterns Jinlin Yang
Partial Order of Patterns Alternating PSPS MultiEffect PSS MultiCause PPS EffectFirst SPS Stricter OneCause SPSS CauseFirst PPSS OneEffect SPPS Response SPPSPS Jinlin Yang
Logical Relation of Patterns Alternating PSPS • For each combination of two events • Decide if they satisfy CauseFirst, OneCause, or OneEffect • Find the strictest pattern Λ MultiEffect PSS MultiCause PPS EffectFirst SPS Stricter Λ Λ Λ OneCause SPSS CauseFirst PPSS OneEffect SPPS Response SPPSPS Jinlin Yang
Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces Jinlin Yang
Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces Jinlin Yang
Find Strictest Pattern For any two events, determine the strictest pattern satisfied by all traces CauseFirstΛOneEffect MultiCause Jinlin Yang
Implementation Program Inferred Properties Instrumented Program Execution Traces Instrumentation Testing Inference • Automated testing and inference • Instrumentation and properties diff by hand • 900 lines of Perl code • 93 different events, 100 traces, each has 222 events on average: less than an hour Test Suite Candidate Property Patterns Jinlin Yang
Experiments • Hypotheses • We can automatically extract interesting temporal properties. • Differences of temporal properties among multiple versions can reveal interesting things. • Target programs • Tour bus simulator (8 student submissions) • OpenSSL (0.9.6, 0.9.7-0.9.7d) • Simplified producer-consumer (in the paper) Jinlin Yang
Tour Bus Simulator • Bus and each passenger is a separate thread • Assignment in Fall 2003 graduate-level course (before we started this project) • 8 submissions from the instructor Jinlin Yang
Testing N, the number of people C, the capacity of the bus T, the number of trips Executed each submission with 100 randomly generated inputs, where 20 < C ≤ 40 C+1 ≤ N ≤ 2C 1 ≤ T ≤ 10 >cville_bus –N 2 –C 1 –T 2 Bus waiting for trip 1 Passenger 0 gets in Bus drives around Charlottesville Passenger 0 gets off Bus waiting for trip 2 Passenger 1 gets in Bus drives around Charlottesville Passenger 1 gets off Bus stops for the day Jinlin Yang
Results: Tour Bus Simulator • Differences reveal problems • Bus drives around before all passengers get off • Other problems Jinlin Yang
OpenSSL • Widely used implementation of the Secure Socket Layer protocol • 6 versions [0.9.6, 0.9.7, 0.9.7a-d] • We focus on the handshake protocol Jinlin Yang
Testing • Manually instrumented server • Modified client: advance to a different state with 5% probability • Executed each version of server with 1000 randomly generated clients Jinlin Yang
Inferred Alternating Patterns Documented improvement Race condition Fixed bug 7 alternating patterns same for all versions Jinlin Yang
Partitioning Traces • Correct clients • Properties are exactly same as specified • All versions agree • Faulty clients without server errors generated • Found one possible vulnerability of DOS attack • Segmentation faults • A documented bug fixed since 0.9.7d Jinlin Yang
Summary of Experiments • Useful in program evolution • Reveal interesting changes in OpenSSL • Identify unexpected differences • Revealed bugs • Tour bus: identified faulty implementation • Multiple implementations are rare, but multiple representations are common (i.e. design, model, code) • OpenSSL: partitioning traces Jinlin Yang
Future Work/Research Questions • Automating instrumentation/validation • Can we automatically identify interesting events? • Identifying more useful patterns • Combine data-flow with events • More expressive, more events • Ed will talk more about this • Prioritize results • How to identify interesting differences? • Testing • How do the inferred properties vary with the test suite? Jinlin Yang
Conclusion • Automatically inferring temporal properties is feasible • Even very simple property patterns reveal interesting program properties Jinlin Yang
Thank You! http://www.cs.virginia.edu/ipa This work is funded in part by the National Science Foundation. We thank Marty Humphrey for providing the student submissions, Chengdu Huang for helping OpenSSL, Joel Winstead for discussion and comments on earlier version of this paper, anonymous PASTE reviewers for insightful comments, anonymous CS650 students who wrote the buggy code, and our research group members and CS graduate students at the UVa for helping improve the talk. Jinlin Yang