Fitness Guided Fault Localization with Coevolutionary Automated Software Correction Case Study

1. Objective Add support for multi-objective optimization style fitness calculation to the Fitness Guided Fault Localization (FGFL) System Fitness Guided Fault Localization with Coevolutionary Automated Software Correction Case Study ISC PI: Dr. Daniel Tauritz, Computer Science Co-PIs: Dr. Bruce McMillin, Computer Science & Dr. Thomas Weigert, Computer Science ISC Graduate Student: Josh Wilkerson, Computer Science Missouri S&T Natural Computation Laboratory • Technical Background: • Multi-Objective Optimization • Simultaneously optimize problem objectives • Candidate solutions are given scores for each objective • Non-Dominated Sorting Genetic Algorithm (NSGA): algorithm that FGFL multi-objective approach is based on • NSGA Details: • Solution fitness is proportional to the number of other solutions dominated • Solution S1 dominates S2 if S1 performs at least as good as S2 on all objectives and better on at least one objective • Non-Dominated Front: set of solutions that are not dominated by any other known solutions • Technical Background: • The FGFL System • Fault localization: the most time consuming step in software debugging • Would greatly benefit from automation • FGFL: automated fault localization system exploiting a fitness function for the program • Fitness function: type of objective function used to rate the performance of a solution • Ensemble of techniques used in FGFL, currently consisting of: • Trace Comparison • Trend Based Line Suspicion • Fitness monitor • Results of (active) techniques combined using confidence based voting system • Benefits and Motivation • Most real world problems consist of multiple objectives • Multi-objective fitness calculation allows problems to remain closer to their natural state in the FGFL system • Combining objectives together into a single aggregate fitness function is difficult, as there infinite possibilities • Multi-objective fitness functions avoid having to make tradeoff decisions a priori • The dominance relationship between solutions is stronger than comparison employing an aggregate fitness function • Multi-objective comparison is expected to lead to an improved FGFL test case set • Separate objective scores provide more insight into a solution’s performance • FGFL enhancements and techniques that specifically exploit multiple objective functions • Current and Future Work • Continue development and testing of FGFL enhancements and new techniques exploiting multiple objective functions • Investigate enhancements to existing FGFL techniques to exploit multiple objective functions • Conduct case study using the Coevolutionary Automated Software Correction (CASC) system • Use FGFL to indicate section of buggy code to be corrected • Use both aggregate and multi-objective function • Testing using the Siemens test suite of programs with injected faults Example Front Organization Example Fault Localization Result: Warmer Color Indicates Higher Suspicion • Proposed FGFL Technique: • Cross Objective Testing • Examine execution characteristics of test cases on the non-dominated front with sufficiently unique performance • The closer to the pareto-optimal front, the better • Differences in these characteristics can be employed to provide information on the nature and location of the bug • Can also be used to enhance existing FGFL techniques • Proposed FGFL Enhancement: • Objective Sensitivity Testing • Through testing, objectives effected by a bug could be identified • Identifying objectives sensitive to the bug will allow for more targeted testing • Ideally resulting in more accurate results • Removing objectives from consideration would result in a significant search space reduction