Efficient Runtime for Detecting Defects - QVM Analysis Approach

1. An Efficient Runtime for Detecting Defects in Deployed Systems QVM Matthew Arnold Martin Vechev Eran Yahav IBM Research

2. Motivation:Dynamic Analysis for Debugging

3. Motivation:Dynamic Analysis for Debugging Testing High overhead tolerable Deep properties relating to program correctness

4. Motivation:Dynamic Analysis for Debugging • Production • Low overhead is mandatory • Very limited information Testing High overhead tolerable Deep properties relating to program correctness

5. Motivation:Dynamic Analysis for Debugging Testing • High overhead tolerable • Production • Very limited information • Production • Low overhead is mandatory Testing Deep properties relating to program correctness

6. But Why Modify the VM? VM Disadvantages Portability Complexity Why not use bytecode inst., JVMTI, aspects, java.lang.Instrument ??? VM Advantages VM only information Free bits in object header Can walk the heap if we desire (GC) Performance Exploit dynamic optimization technology Ease of deployment No “install process”. Just set command line flag

7. New Overhead Philosophy Traditional dynamic analysis If I use your analysis, how much overhead will it have? QVM: user specifies an overhead budget “I am willing to tolerate an X% slowdown” Goal: give user as much useful information as possible May miss errors But enables somechecking in scenarios where it is currently infeasible

8. Contributions • Overhead manager (OHM) • Adapts analyses to meet user-specified overhead budget • Dynamic analyses checking correctness properties • Typestate property checking • Object-centric sampling • Heap probes / assertions • Java assertions • QVMI • Overhead aware interface for medium-granularity VM events All implemented and evaluated IBM’s J9 JVM

9. QVM Architecture Application typestate specs violations report typestate client heap probes client assertions client Clients QVMI observed overhead event filters event callbacks Execution Engine overhead budget OHM adjust sampling rates VM Core QVM

10. QVMI: The QVM Interface Analysis Client event filters JVMTI event callbacks VM Execution Engine • Profiling interface • Similar to JVMPI/JVMTI • Method calls, allocations, etc • Key Difference: event filtering

11. QVMI: The QVM Interface Analysis Client Analysis Client event filters event filters JVMTI QVMI event callbacks event callbacks VM Execution Engine Execution Engine • Profiling interface • Similar to JVMPI/JVMTI • Method calls, allocations, etc • Key Difference: event filtering

12. QVMI: The QVM Interface When compiling a method JIT queries QVM clients “Does invocation of method foo() require a call back?” If not, no callback is compiled into code Ensures no overhead for uninteresting events

13. Overhead Manager (OHM) QVMI observed overhead event filters event callbacks Execution Engine specified overhead OHM adjust sampling rates VM Core Monitoring: measure overhead incurred by clients Sampling strategy: events callbacks have adjustable sample rate Controller: adjusts sample rate based on measured overhead

14. Overhead Manager Challenges Fine grained timers critical Read cycle counts via rdtsc instruction Must have notion of “total application time” Interactive apps We use Linux getrusage()to get cpu time Issues for multi-threaded apps Details in paper Analyses must be able to be “turned off” OK to miss bugs But must not produce meaningless results

15. Maximizing Sampling Coverage Randomly distributed sampling can produce poor results Execution frequency Code eventA (…) eventB (…) eventC(…)

16. Maximizing Sampling Coverage Origin-specific sampling Execution frequency Sample Rate Code 1/1 eventA (…) 1/1 eventB (…) 1/100 eventC(…)

17. QVM Client 1: Typestate Property Checker b * dispose* | release* disposed else undisposed Objectallocation err Objectdeath *

18. Typestate Property Checker Simple to implement via QVMI Events used Object Allocation, method invocation, object death Sampling typestate is problematic Ex: File Open  Close High problem of sampling close but not open Solution: object-centric sampling

19. Object Centric Sampling … tracked tracked T t = new T() … • Tracked objects marked using bit in object header • Bit checked before executing callbacks

20. Client 2: Heap Probes • Heap Probes • Allow programmer to query properties of the heap • isShared(Object o1) • Do two or more heap objects point to o1 • isThreadOwned(Thread t, Object o) • Is o reachable from only thread t only • Uses components of a parallel GC to evaluate heap queries • Worst case: requires traversal of entire heap • Probe sites automatically sampled by overhead manager • Performs GC in process

21. Use Case Example: Azureus Over 160 million downloads

22. Azureus Resource Leaks • Typestate checker for undisposed GDI resources • Actual QVM report: QVM ERROR: [Resource_not_disposed] object [0x98837030] of class [org/eclipse/swt/graphics/Image] allocated at site ID 2742 in method [com/aelitis/azureus/.../ListView.handleResize(Z)V] died in state [UNDISPOSED] with last QVM method invoked [org/.../Image.isDisposed()Z].

23. Azureus Resource Leaks OS Resources imgView OS Resources void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { imgView = new Image(listCanvas.getDisplay(), clientArea); } // … }

24. Possible Fix void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { if(imgView != null && !imgView.isDisposed()) { imgView.dispose(); } imgView = new Image(listCanvas.getDisplay(), clientArea); } // ... }

25. Possible Fix void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { if(imgView != null && !imgView.isDisposed()) { assert(!QVM.isShared (imgView)); imgView.dispose(); } imgView = new Image(listCanvas.getDisplay(), clientArea); } // ... }

26. Experimental Evaluation

27. Overhead Manager: stabilization

28. Overhead Manager

29. Leak Detection Results

30. Sampling coverage (5% budget)

31. Summary • Recap • Adaptive overhead controller • Clients: typestate, assertions, heap probes • QVMI • Found and fixed bugs several real applications • Future Work • Improve efficiency of heap assertions • Concurrent or incremental evaluation • Overhead manager • Tighter overhead guarantees

32. Related Work • Much related work in paper • Typestate and leak detection • Aspect Oriented Programming • “Monitoring Oriented Programming” • Heap analysis tools • Real time garbage collection

33. The End