170 likes | 342 Views
Trustworthy Software Everywhere. Tao Xie Automated Software Engineering Group Department of Computer Science North Carolina State University https://sites.google.com/site/asergrp/ . Trustworthy Software Everywhere. Static Verification Problem: API properties are not available
E N D
Trustworthy Software Everywhere • Tao Xie • Automated Software Engineering Group • Department of Computer Science • North Carolina State University • https://sites.google.com/site/asergrp/
Trustworthy Software Everywhere • Static Verification • Problem: API properties are not available • E.g., fopen’s return needs to be NULL-CHECK • Solution: mining API properties from client code • Dynamic Verification (a.k.a. Software Testing)
Searching + Mining Supported by NSF CSR and ARO
PARSEWeb[ASE 07] Source object type & Destination object type Method-invocation sequence PARSEWeb Development Assisted by Searching+Mining MAPO [ECOOP 09] API method Frequent subsequences of API methods SpotWeb[ASE 08] Framework hotspots/coldspots
Verification Assisted by Searching+Mining • Detect deviant behavior as bugs in programs • Neglected-condition bugs [ASE 09] • Exception-handling bugs [ICSE 09] • Error-handling bugs [FASE 09] • API-sequencing bugs [ESEC/FSE 07]
Mining Text in Software Engineering • Detect duplicate bug reports [ICSE 08] • Identify security bug reports [MSR 10] • Mine resource specifications from Javadoc[ASE 09, Best Paper Award, SIGSOFT Distinguished Paper] • javax.resource.cci.Connection • createInteraction():“Creates an interaction associated with this connection.” • getMetaData():“Gets the information on the underlying EIS instance represented through an active connection.” • close():“Initiates close of the connection handle at the application level.” Supported by IBM Jazz Award
? = Software Testing Problems + Expected Outputs Test inputs Program Outputs Test Oracles • Test Generation • Generating high-quality test inputs (e.g., achieving high code coverage) • Test Oracles • Specifying high-quality test oracles (e.g., guarding against various faults)
Test Generation • Human • Expensive, incomplete, … • Brute Force • Pairwise, predefined data, etc… • Random: • Cheap, Fast • “It passed a thousand tests” feeling • Dynamic Symbolic Execution: Pex, CUTE,EXE • Automated white-box • Not random – Constraint Solving
Dynamic Symbolic Execution Choose next path • Code to generate inputs for: Solve Execute&Monitor void CoverMe(int[] a) { if (a == null) return; if (a.Length > 0) if (a[0] == 1234567890) throw new Exception("bug"); } Negated condition a==null F T a.Length>0 T F Done: There is no path left. a[0]==123… F T Data null {} {0} {123…} Observed constraints a==null a!=null && !(a.Length>0) a!=null && a.Length>0 && a[0]!=1234567890 a!=null && a.Length>0 && a[0]==1234567890 Constraints to solve a!=null a!=null && a.Length>0 a!=null && a.Length>0 && a[0]==1234567890
Challenges of DSE • Loops • Fitnex [DSN 09] • Generic API functions e.g., RegEx matching IsMatch(s1,regex1) • Reggae [ASE 09-sp] • Method sequences • MSeqGen [ESEC/FSE 09] • Environments e.g., file systems, network, db, … • Parameterized Mock Objects [AST 09, ASE 10-sp] Opportunities • Regression testing [ICSE 09-nier] • Developer guidance (cooperative developer testing) Supported by NSF SoD, NSF SHF, NSF CAREER, Microsoft Research Award
NCSU Industry Tech Transfer • Loops • Fitnex [DSN 09] • Generic API functions e.g., RegEx matching IsMatch(s1,regex1) • Reggae [ASE 09-sp] • Method sequences • MSeqGen [ESEC/FSE 09] • Environments e.g., file systems, network, db, … • Parameterized Mock Objects [AST 09, ASE 10-sp] Applications • Test network app @Army division, Fort Hood, Texas • Test DB app of hand-held medical assistant device@FDA • Test .NET base libraries @Microsoft
PexVisual Studio Power Tool • Download counts (20 months)(Feb. 2008 - Oct. 2009 ) • Academic: 17,366 • Devlabs: 13,022 • Total: 30,388
Trustworthy Software Everywhere • Various countries/regions • Software internationalization • Locating constant strings to translate [ICSE 09, FSE 10] • E.g., translating Megamek (a realtime strategy game) • Various programming languages • PL translation • E.g., translating Java to C# [ICSE 10]
Trustworthy Software Everywhere • Various types of software • Database applications [ASE 10-sp] • Network/file-system applications [AST 09] • Game applications [ICSE 09] • Cyber-physical systems (power grid, medical device software, …) • Could applications • Social network applications • … Supported by NSF SHF, NSF CAREER
Trustworthy Software Everywhere • Various types of quality attributes • Functional correctness • Security (NIST/Fermi Lab collaboration) • Testing/verification of access control policies [WWW 07, ACSAC 08, SIGMETRICS 08, IEEE TC 10] • Testing/verification of firewall policies [SRDS 08/09, LISA 10 Best Student Paper] • Identification of security bug reports [MSR 10] • Attack generation • Performance • IBM RTP collaboration on Rational Performance Tester • … Supported by NSF CyberTrust, NIST, IBM Faculty Awards
Trustworthy Software Everywhere http://people.engr.ncsu.edu/txie/ https://sites.google.com/site/asergrp/