Supporting Dynamic Communication by Software Development Histories

1. Supporting Dynamic Communication by Software Development Histories Katsuro Inoue Makoto Matsushita Kei Sasaki Osaka University

2. Classification of SE Technologies by Target Scale Mega Software Engineering Dync Symposium 2004

3. Mega Software Engineering MSE • Targets many projects • A new concept but not a new technology itself • Collection of key technologies already existing and emerging • Distributed environment and data sharing • Analysis and data mining • Project monitoring and controlling • Scalable computing • ... • Use advances of hardware performance, e.g., network bandwidth, CPU clock, memory space, disk capacity, ... • Software engineering technology should share in advances of hardware, which is mainly used for multimedia, grid, simulator, ... Dync Symposium 2004

4. Characteristics of MSE • Experience and knowledge of individual developer or project are collected, refined as assets, and reused in community • Non-structured flat dynamic community for information sharing • View from the organizational benefits may be directly obtained • Open sourcecommunity is an example case of MSE Dync Symposium 2004

5. Analysis Related Organization Software Development Organization Concept of Empirical Project Monitor EPM Internet Public Domain Software Open Source Project Collection Improvement

6. EPM（developing） Code clone detection Component search Metrics measurement Project categorization Cooperative filtering Product data archive (CVS format) Process data archive (XML format) Format Translator Format Translator Format Translator Format Translator Versioning (CVS) Mailing (Mailman) Issue tracking (GNATS) Other tool data Managers Project x Project y Corporate Source GUI Project z . . . Developers

7. S P A R S-J Software Product Archiving, Analyzing and Retrieving System for Java Analyzer and Evaluator Component Collector Internet / Corporate Repositories Query Handler Component Archive SPARS-J Software Component Searcher Dync Symposium 2004

10. Query Word = Source code Commit log Keyword, Time File name Developer name Query Word Search Result Commit log Keyword Time File name Developer name Similarity Time File name Developer name Source code Related Files /Data CVS information E-mail information Data Display Record System CGI-Main Search result File name Developer name Time Source code Developer name Time Topics Search Result Lexical analysis tool Token compare tool CVS Info DB Fusion info DB E-mail InfoDB Token Code DB Fusion info Create tool CVS info Create tool E-mail info Create tool DB Create tool CoxR CoxR (Web Server) CoxR user Data Display Record System CGI-Main Lexical analysis tool Token compare tool CVS Info DB Fusion info DB E-mail InfoDB Code DB CoDS SPxR Fusion info Create tool CVS info Create tool E-mail info Create tool DB Create tool E-mail Archive CVS Repository

11. Extension of CoxR to Fit to DynC Unveil development community by repositories • Extracting knowledge (5W1H) from SCM repository, email archives, bug tracking system • Detecting relationship among persons, products, email, … • Find communities according to the network of relationship. Dync Symposium 2004

12. Classify Knowledge Knowledge: intention + task • Intention … of software development • Discussion of whole system architecture • Problem with a new code Development policy • Task • Results of development • Features enhanced • Actual software changes Requests for changes New features Dync Symposium 2004

13. Simple Example Development history Kernel panic happened when I changed ipfw(8) rules, why? On certain situation, a race condition will happen while tcp_syncache() calls outside functions. No more panic even if syncache_drop() is called just aftter syncache_respond(). administrator Bug overview Discussion A Bug report on the kernel panic 2003/05/22 using memory after freed in tcp_syncache ・・・・ 2003/05/28 src/sys/netinet/tcp_syncache.c 1.5.8 Bug type: software way to happen: ifconfig lo0 192.168.2.1 alias ipfw 50 add fwd 192.168.2.1,23 tcp from any to 192.168.2.1 some time ... description: Error…. Bug fix 2003/06/14 src/sys/netinet/tcp_var.h 1.56.2.12 Discussion B Patch needed to src/sys/netinet/tcp_syncache.c and tcp_var.h, however, it may cause unwanted side-effect to others. Workaround This problem is not related to ipfw(8); see previous email threads for how to fix. Bug tracking system Mailing list CVS

14. Knowledge Map to Intention/Task Don’t panic even if syncache_drop() is called just aftter syncache_respond(). Don’t panic even if syncache_drop() is called just aftter syncache_respond(). On certain situation, a race condition will happen while tcp_syncache() calls outside functions. On certain situation, a race condition will happen while tcp_syncache() calls outside functions. Kernel panic was happen when I changed ipfw(8) rules, why? Kernel panic happened when I changed ipfw(8) rules, why? Type of bug: Software-bug How to reproduce: ifconfig lo0 192.168.2.1 alias … Description: …. Type of bug: Software-bug How to reproduce: ifconfig lo0 192.168.2.1 alias … Description: …. Patch needed to src/sys/netinet/tcp_syncache.c and tcp_var.h, however, it may cause unwanted side-effect to others. Patch needed to src/sys/netinet/tcp_syncache.c and tcp_var.h, however, it may cause unwanted side-effect to others. Intention：Development directions, problems, and reasons This problem is not related to ipfw(8); see previous email threads for how to fix. This problem is not related to ipfw(8); see previous email threads for how to fix. Task：What’s happened, actual code changes Source code (diffs)

15. Contents of Intention/Task Intention Task Filename Revision Tag, Date • Source code • Inserted • Deleted CVS Log message Keywords Filename Revision Subject Mail body E-mail Submit date Category Bug type Overview How to repeat Filename BTS ID Severity Status Description How to fix Last updated date Update history Closed date BTS Dync Symposium 2004

16. Intention/Task Networks How do I know how to fix the failure in feature A? It seems that that files may have a bug… Extract intention/Task Sets, which are related to current topic Task Set B: modify similar files Feature A intention set of feature B Similar modifications to files CVS β debugger E-mail g CVS α E-mail e BTSλ Similar BTS entries CVS θ BTS f E-mail b BTS a Log d BTS ζ E-mail c CVS γ CVS φ intention set of similar bugs CVS δ Task Set C: similar features of A CVS ψ CVS ν CVS ω Dync Symposium 2004 Task Set D: similar contents

17. Query Word = Source code Commit log Keyword, Time File name Developer name Query Word user Results System Overview Web Server Source code keywords Log Filename Developer name Intention and Task Developer (Group),Role Query Word results System Control System Control CoxR Role Develop Group intention Relations task Role DB Role DB Process DB Task DB Task DB Rel. DB Rel. DB Intention DB Intention DB Extract info 情報 抽出 Knowledge CVS Analyzer E-mail Supporting DC System Bug Tracking System Dync Symposium 2004

18. Conclusion • Mega Software Engineering • EASE, SPRAS-J, CoxR • DynC as a part of Mega Software Engineering • Extensions of CoxR • Knowledege: Intentions and Tasks • Extract and Grouping • Developers Role • Extracting dynamic community from the network organized with intentions and tasks • System is under implementation Dync Symposium 2004

19. Dync Symposium 2004

20. Intention • Log message • It tells us about “hat kind of changes has been made, or why this change is needed, toward the files committed at the same time. • Subject・mail body • Discussions between developers on future directions, a certain source code, how-to-fix the bug. • BTS entry • The type of bug, how-to-repeat the bug, what failures are there, what change is desired, are described clearly. Dync Symposium 2004

21. Task • Source code • Actual changes to software itself • Easy to know what portion was changed, by checking the delta against previous revision. • Fix section (in each BTS entry) • It records modification histories and what changes will fix the bug. Dync Symposium 2004

22. Relationship between developers • Detect core development group of the software • Developers who join similar intention Sets • Joining many discussions • Developers who join similar Task Sets • Developer who joins the same products • Define roles of developers Manager Controlling project itself Total programmer Modify many portions of software Programmer Modify several portions of software Tester Submit many change requests Debugger Mainly fixing bugs Administrator Managing stable products Dync Symposium 2004

23. Grouping intention/Task Information in a same file Same revision Task log code Intention Task How-to repeat Same BTS entry fixes Information with a same filename, revision, BTS ID Discussion on fixes Task fixes E-mail Intention Discussion on architecture code Modification request How-to repeat log code Work- around Modification according to workaround Dync Symposium 2004

24. Relationships between intention/Task • Each intention/task is spread across some systems, such as CVS, email, etc. • Also, relation between each intention/task is also scattered about repositories. • May contain: • Development log • Email • BTS entry • May contain: • Source code • Bug fixes Task code log Intention Task Email fixes BTS Intention Set Task Set Dync Symposium 2004

25. Extracting developers concerned I’d like to know who implemented the feature, and who fixed some bugs Ok, maybe that’s all information I may consult, but where to check first? Task Set B: modify similar files intention set of feature B CVS β E-mail g CVS α E-mail e BTSλ CVS θ BTS f E-mail b BTS a Log d BTS ζ E-mail c CVS γ CVS φ intention set of similar bugs CVS δ CVS ψ Developers of managing bugfixes Task Set C: similar features of A Main feature developers CVS ν CVS ω Dync Symposium 2004 Task Set D: similar contents

26. Narrowing networks down I’d like to check similar features’ development histories and bugfixes… Task Set B: modify similar files intention set of feature B CVS β E-mail g CVS α E-mail e BTSλ CVS θ BTS f E-mail b BTS a Log d BTS ζ E-mail c CVS γ CVS φ intention set of similar bugs CVS δ CVS ψ Task Set C: similar features of A CVS ν CVS ω Dync Symposium 2004 Task Set D: similar contents

27. Find what you need on your concerns Development results from the past Past development of the feature CVS φ CVS φ CVS φ’ BTS p Further policies on developing the feature CVS r E-mail h CVS r CVS r CVS γ’ Bugfix information of the feature BTS k E-mail b BTS a Log d BTS ζ E-mail c CVS γ CVS φ intention set of similar bugs After identifying the developers and knowledge automatically, find what you want by traversing information selected. Task Set C: similar features of A Dync Symposium 2004

28. Gathering the answer CVS φ CVS φ’ CVS φ CVS φ’ BTS p CVS φ E-mail h E-mail h CVS γ’ CVS φ CVS γ” CVS φ CVS γ’ BTS k E-mail b BTS a Log d BTS ζ E-mail c CVS γ E-mail c CVS γ CVS φ CVS φ intention set of similar bugs Gathering the “how-to fix” appropriately, from the Task Set of similar feature Task Set C: similar features of A Dync Symposium 2004

29. System implementation • Data extraction • Extracting intentions and Tasks • Data extraction • Extracting intentions and Tasks • Data analysis • Intention analysis • Grouping Intention Sets • Task analysis • Grouping Task Sets • Data integration • Relationship among Sets • Role Analysis • Core development group and roles Create DBs • Database managing unit Dync Symposium 2004