1 / 25

Improved Fitness Functions for Automated Program Repair

Improved Fitness Functions for Automated Program Repair. Zachary P. Fry. Improved Fitness Functions. Automatic program repair can fix bugs. GenProg. Bugs. Fixes. Improved Fitness Functions. Automatic program repair can fix bugs. GenProg. Bugs. Fixes. Fitness Functions.

osborn
Download Presentation

Improved Fitness Functions for Automated Program Repair

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improved Fitness Functions for Automated Program Repair Zachary P. Fry

  2. Improved Fitness Functions Automatic program repair can fix bugs. GenProg Bugs Fixes

  3. Improved Fitness Functions Automatic program repair can fix bugs. GenProg Bugs Fixes Fitness Functions

  4. Improved Fitness Functions • The current fitness model is imprecise • Ideas: • Not all test cases are created equal • Test cases may not describe all relevant program behavior • Different types of bugs might benefit from different kinds of fixes • We propose to address the naivety of the current fitness representation.

  5. Fitness Distance Correlation • “Quantifying the extent to which a GA fitness function approaches an ideal of heuristic search”1 • Informally, does a given fitness function produce values that correlate with some grounded notion of “closeness to a fix”? 1) T. Jones and S. Forrest. Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In International Conference on Genetic Algorithms, pages 184–192, 1995.

  6. Improved Fitness Functions • Measuring proximity to a fix • Insert, delete, and swapping lines in the program FIX Fix d(135) i(251,205) i(774,111) s(598,324) NO FIX

  7. Improved Fitness Functions • Measuring proximity to a fix • Insert, delete, and swapping lines in the program FIX Fix d(135) i(251,205) i(774,111) s(598,324) M1 i(251,205) i(774,111) s(598,324) d(63) NO FIX

  8. Improved Fitness Functions • Measuring proximity to a fix • Insert, delete, and swapping lines in the program FIX Fix d(135) i(251,205) i(774,111) s(598,324) ✓ ✓ ✓ ✗ 75% M1 i(251,205) i(774,111) s(598,324) d(63) NO FIX

  9. Improved Fitness Functions • Measuring proximity to a fix • Insert, delete, and swapping lines in the program FIX Fix d(135) i(251,205) i(774,111) s(598,324) ✓ ✓ ✓ ✗ 75% M1 i(251,205) i(774,111) s(598,324) d(63) M2 d(84) s(844,265) i(774,111) i(735,431) NO FIX

  10. Improved Fitness Functions • Measuring proximity to a fix • Insert, delete, and swapping lines in the program FIX Fix d(135) i(251,205) i(774,111) s(598,324) ✓ ✓ ✓ ✗ 75% M1 i(251,205) i(774,111) s(598,324) d(63) ✗ ✗ ✓ ✗ 25% M2 d(84) s(844,265) i(774,111) i(735,431) NO FIX

  11. Improved Fitness Functions • The current model of fitness does not correlate well with proximity to a fix (0.145). • Hypothesis: By taking into account previously unused information about test cases, bugs, and fixes we can better inform the evolutionary bug fixing process to fix bugs faster and more often.

  12. Improved Fitness Functions • Approach: weight test cases based on known fixes FIX NO FIX M1 Test Case 1 M1 Test Case 2 M2 M2 M3 M3 M4 M4

  13. Improved Fitness Functions • Approach: weight test cases based on known fixes FIX NO FIX M1 Test Case 1 M1 Test Case 2 M2 M2 M3 M3 M4 M4

  14. Improved Fitness Functions • Approach: weight test cases based on known fixes FIX NO FIX M1 Test Case 1 M1 Test Case 2 M2 M2 M3 M3 0.2 0.8 M4 M4

  15. Improved Fitness Functions • Evaluation: • How much can we speed up fixes? • Computational time and monetary cost • Preliminary results • How many more bugs can we fix? • Fraction of previously unfixed bugs • Future work

  16. Preliminary Results • For a sample of 15 bugs from one program, 31.3% of test cases show no correlation with actual fitness (closeness to a fix)

  17. Preliminary Results • Some test cases are over 23x more correlated with actual fitness than others • Suggests an adequate weighting scheme using machine learning could fix more bugs, faster • This workand additional efforts to investigate other strategies for improving fitness functions are ongoing.

  18. Applicability • When might this work? • Programs with expensive test suites – e.g. Php(12,000+) • When there is heavy overlap between test cases • Test suites/cases that fail to specify the bug • Assumptions? • Presence of historical bug fix data to mine • Test suites do not evolve drastically from bug to bug • Bugs for a given program are related on some level

  19. Goals • By providing GenProg a better signal for mutants’ fitness we hope to: • Better direct the search – arrive at fixes faster, lowering cost (up to 38%) • In the limit, find more fixes for previously unfixed bugs

  20. Goals • By providing GenProg a better signal for mutants’ fitness we hope to: • Better direct the search – arrive at fixes faster, lowering cost • In the limit, find more fixes for previously unfixed bugs Questions?

  21. Synchronicity Bug Example • int withdraw(intamt){ • if(total_amt >= amt){ • total_amt-=amt; • return 1; • } • return 0; • }

  22. Synchronicity Bug Example • int withdraw(intamt){ • if(total_amt >= amt){ • total_amt-=amt; • return 1; • } • return 0; • } GLB_LOCK=0

  23. Synchronicity Bug Example • int withdraw(intamt){ • if(total_amt >= amt){ • total_amt-=amt; • return 1; • } • return 0; • }

  24. Synchronicity Bug Example • int withdraw(intamt){ • lock(); • if(total_amt >= amt){ • total_amt-=amt; • unlock(); • return 1; • } • unlock(); • return 0; • } GLB_LOCK=1 GLB_LOCK=0

  25. Synchronicity Bug Example • int withdraw(intamt){ • lock(); • if(total_amt >= amt){ • total_amt-=amt; • return 1; • } • return 0; • } GLB_LOCK=1

More Related