Towards Supporting On-Demand Virtual Remodularization Using Program Graphs

1. Towards Supporting On-Demand Virtual Remodularization Using Program Graphs David Shepherd, Lori Pollock, and K. Vijay-Shanker University of Delaware

2. Overview and Rationale //Activates the tool for the given view. //Turn on dragtool by adding.... public void Tool.activate() undo action update drawing on-demand remodularization save drawing activate tool action oriented system object oriented system

3. Overview and Rationale public interface Storable{... //Store the fields in a file.... undo action Key Insight: Programmers leave natural language clues that reference concerns public void Circle.save() update drawing activate tool save drawing action oriented system object oriented system

4. State of the Art in Feature Location/Aspect Mining Low-level Clues • Dynamic Information • [Wilde ICSM 00], [Michail ICSE 02], [Zhao ICSE 04], [Breu ASE 04], [Ceccato ICSM 05]... • Program Structure • FEAT, Sextant, JQuery, [Robillard FSE 05]... • Information Retrieval • [Marcus PhDThesis 03], [ICSE 03] • Lexical Information • Aspect Mining Tool, Aspect Browser Complementary Foundations

5. A Motivating Example Recipe Writing Factory Recipe for Vegetable Soup • Place asparagus and onion in a saucepan with 1/2 cup vegetable broth. Bring the broth to a boil, reduce heat and let simmer until the vegetables are tender. • Reserve a few asparagus tips for garnish. Place remaining vegetable mixture in an electric ... Procedural instructions are easy to read

6. Choosing a Dominant Decomposition Recipe Writing System 1.0

7. Choosing a Dominant Decomposition Recipe Writing System 1.0 • asparagus • Place asparagus in a saucepan. • Reserve a few asparagus tips for garnish • Garnish with reserved asparagus tips. • flour • Sprinkle flour into the butter. Do not let the flour brown. • butter • Melt butter in the pan that was used for simmering the asparagus and onions…stir. • salt • Sprinkle salt into the butter. • soy milk • Stir the milk into the saucepan. • yogurt • Whisk yogurt into the mixture OO decomposition provides many benefits Recipe for Vegetable Soup • List ingredients = {asparagus, onion...} • Bowl b = new Bowl(); • while( ingredients.hasNext){ • Ingredient ingred = ingredients.next(); • ingred.place(b); • ...

8. Action-Oriented CCCs Appear • asparagus • Place asparagus in a saucepan. • Reserve a few asparagus tips for garnish • Garnish with reserved asparagus tips. • flour • Sprinkle flour into the butter. Do not let the flour brown. • butter • Melt butter in the pan that was used for simmering the asparagus and onions…stir. • salt • Sprinkle salt into the butter. • onion • Place onion in the saucepan • yogurt • Whisk yogurt into the mixture OOP causes scattered actions

9. Evolution Causes Scattered Actions //Place the Beef into the pan place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p); } Beef place(p) checkIfFitsInPan(p) actuallyPlaceInPan(p) //Make sure the beef fits in the pan, //before Placing the Beef into the pan checkIfItFitsInPan(Pan p){ ... } //Place the beef into the pan actuallyPlaceInPan(Pan p){ ... } Evolution causes more scattered actions

10. Maintenance Tasks Involve Scattered Actions Something goes wrong when I place ingredients in the saucepan. Maintenance tasks require action location

11. How can we find actions in a noun's world? Our Strategy • Extract natural language clues • Query clues and remodularize

12. Extract Clues with NLP //Make sure the beef fits in the pan, //before placing the beef into the pan Beef.checkIfItFitsInPan(Pan p){ ... } //Make sure the beef fits in the pan, //before placing the beef into the pan Beef.checkIfItFitsInPan(Pan p){ ... } //Place the onion in the pan Onion.place(Pan p){ ... } place onion place beef, check beef //Place the beef into the pan Beef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p); } //Place the beef into the pan Beef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p); } //Place the beef into the pan Beef.actuallyPlaceInPan(Pan p){ ... } //Place the beef into the pan Beef.actuallyPlaceInPan(Pan p){ ... } place beef place beef Key Insight:Use language clues to extract actions

13. Query Clues and Remodularize Virtual Source File //Make sure the beef fits in the pan, //before placing the beef into the pan Beef.checkIfItFitsInPan(Pan p){ ... } simple query: place //Place the beef into the pan Beef.actuallyPlaceInPan(Pan p){ ... } //Place the onion in the pan Onion.place(Pan p){ ... } //Place the beef into the pan Beef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p); } From OO code to Action-Oriented code!

14. A Closer Look at the Extraction Process...

15. What to Extract Something goes wrong when I place ingredients into the saucepan. Something goes wrong when I place beef into the saucepan. • information about actions • verbs • direct-objects ex: Place the item in the pan ex2: Place the beef in the pan Our Strategy: extract verb and direct object pairs

16. How to extract clues Two types of extraction class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); } } Extraction from comments Extraction from method signatures

17. Extracting Clues from Comments • Part-of-speech tagging • Chunking • Role assignment Remove an item from a cart Remove<v> an<dt> item<n> from<p> a<dt> cart<n> Remove<v>[an item]<noun phrase>[from [a cart] <noun phrase>] <prepositional phrase> Remove item rule: when a verb is followed by a noun phrase, extract

18. Extracting Clues from Signatures • Process method name • Use hierarchy of rules to classify • Use classification to extract class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); } }

19. Extracting Clues from Signatures • Process method name • Use hierarchy of rules to classify • Use classification to extract class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); } } chop<v> chop<v> [the beef]<noun phrase>

20. Extracting Clues from Signatures Hand Tuned Rule Tree • Process method name • Use hierarchy of rules to classify • Use classification to extract class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); } } Example Classification: method has no parameters and the method name is a verb chop<v>

21. Extracting Clues from Signatures • Process method name • Use hierarchy of rules to classify • Use classification to extract • Example Extraction Rule: • get verb from method name • get DO from class name class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); } } chop beef

22. Storing the Clues Action-Oriented Identifier Graph (AOIG) verb1 verb2 verb3 DO1 DO2 DO3 verb1, DO1 verb1, DO2 verb3, DO2 verb2, DO3 use use use use use use use use source code files

23. Storing the Clues Action-Oriented Identifier Graph (AOIG) chop grill puree onion beets steak chop, onion chop, beets puree, beets grill, steak use use use use use use use use source code files

24. How Accurate is Extraction? Survey of AOIG-Builder's Accuracy Avg. Precision 95% Avg. Recall 83% Good Precision and Recall

25. Costs of Building the AOIGper source file • Space • O(number of methods) • minimal • Time • O(number of methods) • 10 seconds for a large source file with lots of multi-line comments Reasonable cost - can be done incrementally

26. Contributions...

27. Action-Oriented View • High-level CCCs instead of low-level • clear hypothesis of why actions are CCCs • Useful type of CCC • maintainers are looking for actions

28. Contributions (continued) • Action-Oriented View • Combined NLP+Program Analysis (NLPA) • Developed the AOIG for virtual remodularization • Applied AOIG to • Feature Location* • Aspect Mining • Working Set Recovery In this talk, we have shown how to virtuallyremodularize code into actions using the AOIG

29. Questions? save drawing activate tool update drawing undo action action oriented system object oriented system

30. Conclusions • NLP • is sufficiently robust to use in SE tools • NLP + Program Analysis • can extract more exact information from code • can drive SE tools We have shown how to virtuallyremodularize code into actions using the AOIG

31. "Demo" of Current/Future Work Complete Action Location Methodology • Form initial query (verb and DO) • Expand query using recommendations • NLP • Program knowledge • Generate "Concern Flow Graph"

32. Current/Future Work

33. generate

39. PowerGrid.timePassing() PowerGeneration.constructor PowerGrid.powerSurplus PowerGeneration.getGeneratedPower() PowerSource.getCurrentPower() calls StandardPowerSource.getCurrentPower() SolarPowerSource.getCurrentPower() PowerSource.getCurrentPower()

46. UnfoldAll.foldOneStage() UnfoldAll..foldStageN() UnfoldAll..foldAll() UnfoldAll.setFolded() MindMapActions.setFolded() ControllerAdapter.setFolded() ToggleFoldedAction.setFolded() ToggleFoldedAction.createSingleFoldAction() FoldActionType.setFolded() NodeAction.setNode() FoldActionTypeImple.setFolded() NodeActionImple.setNode()

47. Initial Conclusions of Current Work • Actions often appear as CCs in real code • Semantic queries over language clues can find actions • helps with mapping • Result sets • easier to understand when presented as graph

48. User Feedback DIRECTIONS: • Place asparagus and onion in a saucepan with 1/2 cup vegetable broth. Bring the broth to a boil, reduce heat and let simmer until the vegetables are tender. • Reserve a few asparagus tips for garnish. Place remaining vegetable mixture in an electric blender and puree until smooth. • Melt butter in the pan that was used for simmering the asparagus and onions. Stir while sprinkling flour, salt, and pepper into the butter. Do not let the flour brown. Allow the mixture to cook only 2 minutes. Stir in remaining 1 1/4 cups vegetable broth and increase the heat. Continue stirring until the mixture comes to a boil. • Stir the vegetable puree and milk into the saucepan. Whisk yogurt into the mixture, followed by lemon juice. Stir until heated through, then ladle into bowls. Garnish with reserved asparagus tips. Sprinkle with Parmesan cheese if desired. Something goes wrong when I am sprinkling things into the butter.

49. Recipe Developer, Fix it

50. Recipe Storage System: 2.0 more complex system, actions are scattered further Onion place(Pan p) Person place(List ingredients, Pan p) Placeable place(Pan p) Vegetable place(Pan p) Pepper place(Pan p) Beef place(Pan p)