70 likes | 221 Views
Continuous Systematic Testing of Evolving Software. Regression Testing. Mary Jean Harrold , Darko Marinov , Stephen Oney, Mauro Pezze , Adam Porter, John Penix , Per Runeson, Shin Yoo. Open Problems. Cost/benefit Which test cases are worth executing and when?
E N D
Continuous Systematic Testing of Evolving Software Regression Testing Mary Jean Harrold, Darko Marinov, Stephen Oney, Mauro Pezze, Adam Porter, John Penix, Per Runeson, ShinYoo
Open Problems • Cost/benefit • Which test cases are worth executing and when? • Cost for setup vs execution, creating configurations vs execution etc • Non-functional testing; levels of testing • Data infrastructure • Which data is needed to select test cases systematically? • For today’s methods? For tomorrow’s? • Technical solutions • Selection/prioritization/extension/deletion • Repairing test cases and oracles • Anonymizations and retesting, Concurrency retesting
Promising direction • Gathering and using information that is there already: • changes, code, test cases, executions, configurations, defect reports…
Collaboration • Baseline by sharing previous research • Start by two recent reviews (Yoo, Runeson) • Apply techniques to industry and OSS data • Run ”shadow projects” in multiple domains
Paper titles [1] ”Introducing countinous systematic testing of evolving software” IEEE SW 2010 • Guidance to bits and pieces that need to be in place to get it to work. Cases: ABB, Microsoft, Google [2] ”Cost in the loop of countinous systematic testing of evolving software”, IEEE TSE 2014 • Cost model based test selection
Funding Project:Cost in the loop of Countinous And Systematic Testing of evoLving Software - CASTLeS MARK HARMAN HENRY MUCCINI WOLFRAM SCHULTE TAO XIE