1 / 31

Mock Object Creation for Test Factoring

Improve speed of error detection in Eclipse IDE with mock object creation for test factoring. Learn about factored tests, advantages, and change language violation detection. Enhance testing process automation and developer feedback.

aconley
Download Presentation

Mock Object Creation for Test Factoring

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mock Object Creation for Test Factoring David Saff, Michael D. Ernst MIT CSAIL PASTE, 2004 June

  2. 30 secs 1 sec / plug-in test Set up Eclipse Motivation • Continuous testing plug-in for the Eclipse IDE* • Test suite: • Problem: find out about errors faster • Solution: mock objects to replace Eclipse framework * Saff, Ernst, ETX 2004: Continuous testing in Eclipse

  3. Outline • Mock objects introduced • Test factoring introduced • Mock object creation for test factoring • Conclusion

  4. Outline • Mock objects introduced • Test factoring introduced • Mock object creation for test factoring • Conclusion

  5. Unit test for plug-in Provided Checked Plug-in

  6. System Test for plug-in Provided Checked Plug-in Eclipse

  7. Provided Checked Plug-in Provided Provided Checked Checked Mock Objects replacing Eclipse Provided Checked Unit Test with Mock Object • A mock object: • provides part of the functionality of the original object(s) • is focused on allowing the test to proceed

  8. Mock objects for our example • Using a debugger, determined: • 147 static calls from plug-in to framework • Defined on 49 classes • 8 callbacks from framework to plug-in • Substantial work to define mock objects. • How well can we automate this process without additional manual effort?

  9. Outline • Mock objects introduced • Test factoring introduced • Mock object creation for test factoring • Conclusion

  10. What is a factored test? • Split a system test into several smaller factored tests that • exercise less code than system test • can be added to the suite and prioritized • Find out about errors faster • embody assumptions about future code changes

  11. Pros and cons of factored tests • Pro: factored test should be faster if system test • is slow • requires an expensive resource or human interaction • Pro: isolates bugs in subsystems • Con: if assumptions about how developer will change the code are violated, can lead to: • false negatives: OK, some delay • false positives: bad, distract developer

  12. Change method order? Replace with equivalent call? db.insertRecord(“bob”, “314”); db.insertRecord(“alice”, “617”); db.insertRecords( “alice: 617, bob: 314” ); Change language Change language: the set of tolerated changes • When change language is violated, factored test must be discarded and re-created • Can detect violation through analysis, or incorrect result. db.insertRecord(“alice”, “617”); db.insertRecord(“bob”, “314”);

  13. Separate Sequential Code: A small catalog of test factorings • Like refactorings, test factorings can be catalogued, reasoned about, and automated Also “Unroll Loop”, “Inline Method”, etc. to produce sequential code

  14. Plug-in Mocked Eclipse Mocked Plug-in Eclipse A small catalog of test factorings Introduce Mock: Original test

  15. Related work Early warning if assumptions hold Developer makes change Correct test results This work (static + dynamic) Produce factored tests Run factored tests Run original tests Binkley ’97 (static) Slice based on change Run factored tests

  16. Outline • Mock objects introduced • Test factoring introduced • Mock object creation for test factoring • Conclusion

  17. Basic Procedure: Trace Capture Tested Realm Params, Returns, Callbacks “Boundary” Mocked Realm MockExpectations

  18. Basic Procedure: code generation • MockExpectations encodes a state machine: MockExpectations 0 DebugPlugin.getDefault() → [object id 347] 1 [object id 347].getLaunchManager() → [object id 78] 2 [object id 78].uniqueLaunchNameFrom(“a”) → “a134” 3

  19. Expanding the change language • Current tolerated change language includes: • Extract method • Inline method • Using static analysis on mocked code, improve the procedure to include: • Reorder calls to independent objects • Add or remove calls to pure methods

  20. 0 0 0 1 1 1 2 2 2 3 3 3 Reorder calls to independent objects • Group objects that share state into state sets • One MockExpectations per state set: MockExpectations A MockExpectations B MockExpectations C

  21. Add or remove pure method calls • Allow reordering, addition, removal of calls to pure methods: MockExpectations 0 DebugPlugin.getDefault() → [object id 347] Accessors [object id 347].getLaunchManager() → [object id 78] [object id 78].uniqueLaunchNameFrom(“a”) → “a134” [object id 78].removeLaunch(“a134”) → NEXT STATE 1

  22. Outline • Mock objects introduced • Test factoring introduced • Mock object creation for test factoring • Conclusion

  23. Future work • Develop a framework for test factoring • Implement the “Implement Mock” factoring • Analytic evaluation of framework • Capture real-project change data* • Measure notification time, false positives • Case studies of test factoring in practice • How do developers feel about the feedback they receive? * Saff, Ernst, ISSRE 2003: Reducing wasted development time via continuous testing

  24. Conclusion • Test factoring can indicate errors earlier • “Introduce Mock” is an important test factoring for complicated systems • We propose: • Dynamic analysis for building mock objects • Static analysis for increasing the change language • Mail: saff@mit.edu

  25. A small catalog of test factorings • Separate Sequential Test: • [graphic] • Unroll Loop: • [graphic] • Introduce Mock: • [graphic]

  26. Frequent testing is good: • Frequent manual testing in agile methodologies • Frequent automatic testing in continuous testing. • A testing framework should minimize the cost of frequent testing • Suite completes rapidly • First failing test completes rapidly

  27. A1 A2 A3 A2 A1 B2 B1 A3 A2’ A2 A1’ B2’ B1’ A3’ A1 B2 B1 … Getting faster to the first failing test • Default: • Test selection: • Test prioritization: • Test factoring: B1 B2 A1 A2 A3

  28. Dynamic, change-independent test factoring • Dynamic: instrument and run the original test • Change-independent: factoring happens before any changes are made. • Requires a hypothesized change language • Binkley ’97: Static, change-dependent test factoring

  29. Automatic test factoring: change-dependence • Change-dependent test factoring: • After tested code is changed, generate new tests with same result as old tests for that change. • Change-independent test factoring: • Before tested code is changed, generate new tests that have the same result as old tests for some set of changes. Better

  30. Automatic test factoring: static vs. dynamic analysis • Static analysis (Binkley ’97) • Analyze code to determine mock object behavior • Well-suited for change-dependent factoring • May fail • without source • when dependent on file system or user interaction • Guaranteed change language may be restrictive • Dynamic analysis (this work) • Instrument and run the original test, gather logs • May run original test after factored test fails

More Related