1 / 27

Presented by Asheq Hamid

Sourcerer: An Internet-Scale Software Repository Sushil Bajracharya Joel Ossher Cristina Lopes Donald Bren School of Information and Computer Sciences University of California, Irvine. Presented by Asheq Hamid. An Example. Find a code snippet where: class “ B ” inherits class “ A”

joy
Download Presentation

Presented by Asheq Hamid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sourcerer: An Internet-Scale Software RepositorySushilBajracharya Joel Ossher Cristina LopesDonald Bren School of Information and Computer SciencesUniversity of California, Irvine Presented by AsheqHamid

  2. An Example Find a code snippet where: • class “B” inherits class “A” • And class “A” has a method named “C” • And class “B” overrides that method “C” class B extends A{ void C (){ // Body of the //Overridden method } class A{ void C (){ // Body of method C }

  3. How to leverage this type of searching? Google code search, Koders , Krugle code search ?? • Textual search : Ignores rich structural information in the code. • Sourcerer provides an infrastructure upon which this type of searching can be implemented.

  4. Outline of the presentation • Sourcerer Infrastructure : How the repository has been created. • Sourcerer Web Services : What service Sourcerer developers are providing on top of this infrastructure. • Application to Existing Tools : How different existing tools can be benefited from Sourcerer repository. • Future Work/Extension : What can be some useful additions to the project.

  5. Key References • E. Linstead, S. Bajracharya, T. Ngo, P. Rigor, C. Lopes, P. Baldi. Sourcerer: Mining and Searching Internet-Scale Software Repositories. Data Mining and Knowledge Discovery 2008 • S. Bajracharya, J. Ossher, and C. Lopes. Sourcerer – An Infrastructure for Large-scale Collection and Analysis of Open-source Code. In Proceedings of the Third International Workshop on Academic Software Development Tools and Techniques (WASDeTT-3), 2010. • Sourcerer Project Website: http://sourcerer.ics.uci.edu/

  6. Infrastructure The Sourcerer infrastructure comprises five major subsystems: • A system to crawl and manage software repositories. • A system to parse and extract features from the code. • A relational database to store the information. • Various tools to mine, search the database. • A Web-based graphical interface.

  7. 1. Crawl and manage software repositories

  8. 2. Parse and extract features from the code A parser has been written on top of Eclipse’s AST parser. Mainly the following information is extracted after parsing: • Entity • Relation • Keyword • Fingerprint Several passes required on the source code to extract all these information.

  9. Entity • PACKAGE • CLASS • INTERFACE • ENUM • ANNOTATION • INITIALIZER • FIELD • CONSTRUCTOR • METHOD • PARAMETER • LOCAL VARIABLE • ARRAY

  10. Relation

  11. Relation….

  12. Keywords Why keywords?? Because they are useful for faster retrieval of search result. How keywords are extracted ? Fully qualified names are broken according to java convention. For example: “quickSort” is broken into two keywords : “quick” and “sort” .

  13. Fingerprints What is fingerprint? code with particular syntactical signatures . Example: • Find a code snippet with three nested loops. • Find a switch statement with seven cases. Fingerprints are useful for structural searches of source code.

  14. Cross project dependency What is cross project dependency? Every project has some external dependencies. These dependencies are typically packaged in jar files and included along with the source code. • Sourcerer keeps track of these dependency files. • In case of a missing dependency file, Sourcerer tries to locate that jar file base upon missing dependency information.

  15. 3. Store information in the database.

  16. 4. Tools created

  17. 5. Web-based graphical interface Can be found at: http://sourcerer.ics.uci.edu/sourcerer/search/index.jsp

  18. Sourcerer Web services: • Code Search • Repository Access • Dependency Slicing • Similarity Calculation Code Search: Input: Acombination of terms and fields. The query language is based on Lucene’simplementation. Output: Aresult set with detailed information on the entities that matched the queries.

  19. An example query ** Find a method with terms ”week” and ”date” in its short name, that returns a ”String” type and takes in argument with the term ”Date” in its name. Corresponding query: short name: (week date) AND entity type: METHOD AND m_ret_type_sname_contents: String AND m_sig_args_fqn_contents: Date

  20. Repository Access Input: Id of (file | entity | relation | comment) Output:The file that contains the id. Dependency Slicing: What is a dependency slice? A dependency slice of an entity is a program (collection of Java source files) which includes that entity as well as all the entities upon which it depends. A dependency slice should be immediately compilable.

  21. Dependency Slicing… Input: One or more entity ids. Output: Azip file containing the collection of sliced/synthesized Java files that the given set of entities depend on. Similarity Calculation Input: An entity id. Output: Alist of other entities that are similar to the input entity. *How the similarity has been calculated is out of the scope of the paper.

  22. Application to Existing Tools 1. Finding better code snippet • Strathcona is a tool that also uses structural information to find code examples. • Its code repository structure is very similar to that of Sourcerer. • The large repository of Sourcerer can help Strathcona searching code in a bigger repository. • Strathcona:

  23. 1. Finding better code snippet….. • Parseweb is a tool that provides example for object instantiation. • It downloads code from google code search for examples of object instantiation. • There is no way to automatically resolve missing dependencies, it uses some heuristics. • Sourcerer can benefit Parseweb a great deal as the external dependencies are automatically resolved before downloading the source code. • Parseweb:

  24. 2. Information mining • SpotWeband CodeWeb • Look for hotspots: From different API ,finds the entities which have most associations to other entities. • Using Sourcerer, hotspots could be detected directly simply by ordering the entities in a jar by the number of incoming relations. 3. Test driven code search • CodeGenie and Code Conjurer, both use the context provided by a test case to formulate queries. • Code Conjurer’s dependency resolution can be empowered by Sourcerer’s automatic dependency resolution ability.

  25. Future Extensions • Currently support only Java. • Add support for other languages. 1. Multiple Language Support 2. Addressing Evolution. • Needs to create separate project for each new version of a project. • Add automated support for adopting new versions without creating a new project each time .

  26. Future Extensions…. • 3. Considering non-code artifacts. • Many code project contains non-code artifacts like data on issues, bugs, documentation, authorship, developer’s activities/ history etc. • Find an approach to connect Sourcerer’s models and services with these non-code artifacts. • 4. Intergrating with other open source platforms. • There are some open-source quality monitoring platforms. • FLOSSmoleproject is a collaborative effort to collect and analyze large amount of open source project data. • Its database contains more project specific metadata. • Sourcerer contains more structure specific data. • Therefore, integrating Sourcererwith FLOSSmolecould widen the scope and impact of both projects

  27. Questions?

More Related