MIDAS: A Design Quality Assessment Method for Industrial Software

1. MIDAS: A Design Quality Assessment Method for Industrial Software • Ganesh Samarthyam • Girish Suryanarayana • Tushar Sharma • Shrinath Gupta • Siemens Corporate Research and Technologies, • Bangalore, • India Software Engineering in Practice Track International Conference on Software Engineering, San Franscisco CA, 2013

2. Agenda MIDAS

3. Motivation and Background • Results from survey of CT DC AA* architects • * Siemens Corporate Technology Development Center Asia Australia MIDAS

4. Why this Survey? What really is the state of practice in CT DC AA? Is there a need for a design assessment method? If so, what requirements should it address? • * http://sqgne.org/presentations/2012-13/Jones-Sep-2012.pdf MIDAS

5. Survey Design 16 Questions • Category 1: To elicit the background information of respondents • Category 2: To understand details on design quality practices followed in their projects • What design aspects they think affects design quality? • What are the current design assessment tools / techniques they use? • How satisfied are they with the results of tools/techniques? • Category 3: Shortcomings of existing design assessments tools and techniques MIDAS

6. Background of Participants and their Projects MIDAS

7. Satisfaction Level with Current Design Assessment Practices How satisfied are you with the existing tools/techniques that you used? 59% of the architects are only “somewhat satisfied” with existing tools/techniques used for design quality assessments! MIDAS

8. Aspects for Achieving Good Design Quality • Insight 1: Since adherence to project-specific constraints and design principles is important for good design quality, a design assessment method should explicitly focus on them. MIDAS

9. Limitations of Existing Design Assessment Practices – 1 • Insight 2: To harness the benefits of both design analysis tools and manual reviews and counter their deficiencies, a design assessment method should synergize the use of tools and manual reviews. MIDAS

10. Limitations of Existing Design Assessment Practices – 2 70% of the participants also said they need more exposure to relevanttools/techniques for design assessments! Insight 3: To address lack of exposure to design assessment tools and techniques relevant for project contexts, a design assessment method should rely on a comprehensive knowledge base. MIDAS

11. Barriers to Quality Improvement via Refactoring • Insight 4: To allow stakeholders to make informed decisions about what changes to adopt, refactoring suggestions for reported issues should be accompanied with change impact analysis. • Insight 5: The design assessment method should report design issues in a manner so as to serve as a strong evidence to support refactoring. MIDAS

12. Agenda MIDAS

13. Related Work – 1 MIDAS

14. Related Work – 2 MIDAS

15. Can we adopt an existing method? • Most assessment methods have not been validated in an industrial context limiting the number of candidates to choose from. • Many useful elements could be borrowed from existing assessment methods; however, based on requirements from the survey, none can be adopted in their current form. MIDAS

16. Agenda MIDAS

17. EMISQ method as the basis for MIDAS MIDAS

18. Why a 3-view model for design problems? EMISQ (and most others) employs an “ility” based model, which provides an incomplete view of design problems; it also does not provide suggestions to address them => we developed a three-view model for design problems Addressing Insight 1 from our survey • Insight-1: “design assessment method that explicitly focuses on the violations of design principles and project-specific constraints provides more relevant and helpful results to stakeholders.” • These principles and constraints can be leveraged to: • Detect their violations in the design • Cast results of design analyzers in the form of violations of principles or constraints • Guide towards a solution to address the violations MIDAS

19. 3-view Model for Design Problems “Design principle” –based view Abstraction “-ility” – view Flexibility Encapsulation Caused due to the violation of Modularity Effectiveness design problem Hierarchy Reusability Impacts design problem design problem “Constraints” –based view Functionality design problem Language & platform design problem Understand-ability design problem design problem design problem Domain Caused due to the violation of Extendibility Framework Architecture 2 causal views 1 effect view MIDAS

20. Illustration of the 3-view model with examples – 1 Example 1: “Cyclic dependencies between classes” • From “-ility” based view: Problem impacts Flexibility, Extendibility, and Reusability • From “design-principle”-based view: Violation of “Acyclic Dependencies Principle”-> “Ordering” -> “Modularity” • The design principle view also suggests the high-level refactoring required: the dependency order should be made acyclic towards improving modularity MIDAS

21. “Cyclic dependencies” as violation of design principles Single Responsibility Principle Classification Grouping Variation Encapsulation Principle Hiding Abstraction Liskov’s Substitution Principle Decomposition Cyclic dependencies between classes Encapsulation Ordering Acyclic Dependencies Principle Modularity Locality Hierarchy Factoring Open-Closed Principle Layering Principle of Least Commitment Substitutability MIDAS

22. Illustration of the 3-view model with examples – 2 Example 2: A class that should – but does not - implement dispose pattern [.NET framework guidelines - a class that holds any “unmanaged resources” should implement the IDisposable interface and define a Dispose method for releasing the resources (i.e., class must implement Dispose pattern)] • From “-ility” based view: Problem impacts Functionality, and Effectiveness • From “constraint”-based view: Violation of the Language and Platform constraint MIDAS

23. Illustration of the 3-view model with examples – 3 Example 3: “Use of recursion” - a design problem in embedded systems where recursion is prohibited • From “ility” based view: Problem impacts Effectiveness, and Functionality. • From “constraint”-based view: Violation of Domain Constraint • Recommended refactoring step is to use iteration instead. Example 4: MVC implementation does not strictly separate Model, View, and Controller roles when it should • From “-ility” based view: Problem impacts Flexibility, and Extendibility. • From “design principle”-based view: Violation of Decomposition -> Modularity. • From “constraint”-based view: Violation of Architectural Constraint (imposed by the MVC style). MIDAS

24. Salient Characteristics of MIDAS MIDAS: Features • “Goal-orientation”: Assessments are focused towards answering business or technical questions relating to design quality • Synergy of manual review and tool analysis: • Judicious mix of manual review and tool-based analysis to help harness their benefits while countering their deficiencies • For refactoring suggestions, manually analyzing change impact of a large set of refactoring steps in a large code base is a herculean task; so this step may employ suitable tools for impact analysis • Stakeholder-centric reporting: Design Quality Report addresses different stakeholder concerns (managers, architects, etc) MIDAS

25. MIDAS Process Steps MIDAS process steps are derived from EMISQ, ISO 14598, and W-Process MIDAS

26. Agenda MIDAS

27. Project - I [Overview and assessment goals] Project overview • A GUI editor meant for creating customized UI controls/components for automation & drives domain • The GUI editor followed MVC (Model-View-Controller) style • Around 40 controls written in C# (.NET platform) • Source code size was 125k eLOC Goalof the design assessment • The project team was aware of the poor design quality of their software and was interested in improving the design quality. • Identify refactoring candidates and suggest potential solutions in the context of their project constraints. • An example of such a project constraint was that the public interfaces should not be modified since there were already many applications that made use of the UI components for development. MIDAS

28. Project - I [Assessment Details] MIDAS: Assessment details Tools used • FxCop, Gendarme, Doxygen, NDepend, Understand, and Clocksharp. • Selected design rules and metrics from these tools and further tailored the rule sets in these tools towards meeting the assessment objective. Manual assessment • Manually filtered tool violations keeping project constraints in mind • In consultation with project architects, identified refactoring candidates and refactoring steps for addressing them Design Quality Report (DQR) • A detailed report with problems presented as violation of design principles, constraints, or both; prioritized refactoring candidates and steps Intensive assessment • 4 person months (in2 calendar months) MIDAS

29. Project - I [Feedback and Lessons Learned] Positive feedback for assessment • Project team was satisfied with the results of the MIDAS assessment • Project team felt that the refactoring suggestions provided by MIDAS were very relevant and useful to their context and enabled them in meeting the objective of design improvement. • Specifically, they felt that some of the refactoring suggestions were very good and helped them better restructure the class hierarchies in their design. Aspects that need improvement • Assessment result was prioritized list of refactoring suggestions but the project team further decided to divide these suggestions on the basis of the effort required and the risk involved for implementing those suggestions • Points to the need for better elicitation of project requirements/goals MIDAS

30. Project - II [Overview] Project overview • A GUI editor for to assist in the configuration of intelligent electronic devices • This editor can be thought of as a Form designer of Visual Studio but for intelligent electronic devices • This editor followed the MVC architectural style • Developed using C#.NET • Size of 32k eLoC MIDAS

31. Project - II [Assessment goals and details] Goal of the design assessment • The project team was facing difficulties in enhancing the existing features of the editor. They suspected that this was due to the poor design quality. So, their objective behind the assessment was to: • discover the general design issues that were making it difficult to perform changes • understand the possible refactoring solutions that could help address those design issues MIDAS assessment details • Same as Project – 1 • Main difference: The assessment for Project-2 specifically focused on reporting issues in terms of violation of design principles in the technical and management summaries (towards addressing the goal). MIDAS

32. Project - II [Feedback and Lessons Learned] Positive feedback for assessment • In addition to the violations of well-known design principles, the lack of conformance of the low level design to the overarching architectural style was also reported • The project team found this non-conformance especially revealing. • The project team also mentioned that they found the categorization of issues as reported in the DQR useful in determining the most critical issues that needed to be addressed. Aspects that need improvement • Project team felt that assessment duration was long and suggested shortening it in future • However, they also added that the periodic delivery of intermediate reports greatly helped in receiving early inputs. MIDAS

33. Project - III [Project Overview and Assessment Goals] Project overview • Healthcare application that aims to provide an integrated workflow solution for delivery of particle beams and their recording and verification • C# .NET project • Source code size 17.5k eLoC Goal of the design assessment • The project team suspected that certain platform constraints were not being followed properly in the design. For instance: • Whether C#.NET standard patterns related to resource acquisition and release were consistently followed • Whether guidelines suggested by the .NET framework for exception handling were consistently followed • Additionally, the project team desired to know possible refactoring solutions keeping the .NET specific framework constraints in mind MIDAS

34. Project - III [Assessment details] Assessment Details • Same as Project – 1 & Project – 2 • Main difference: Focus on .NET framework-specific constraints due to the assessment objectives. • This required tailoring of tool rule-sets relating to guidelines suggested by the .NET framework. • This assessment involved only 1 expert for 2 months • Due to non-availability of experts and time constraints MIDAS

35. Project - III [Feedback and Lessons Learned] Positive feedback for assessment • DQR presented platform constraint violations. • The project team found these reported issues very useful and they incorporated the suggestions for the reported issues in the ongoing release itself. • To prevent re-occurrence they shared DQR with the entire project team and planned a series of trainings. Aspects that need improvement • Low degree of manual review that was performed during the assessment. • Expected since only one expert was involved in the assessment; and it underlines the importance of experts role in MIDAS assessments. • Further lack of effective and powerful tools (that would report more relevant issues, report lesser number of false positives, etc.) due to which a larger investment was needed in manual reviews. MIDAS

36. Agenda MIDAS

37. Lessons Learned MIDAS: Lessons learned "Goal-centric” approach works in practice • Main reason: Project needs for an assessment get captured as assessment goals and they get addressed. Expertise in assessments • Projects themselves have experts knowledgeable in design • Having expertise in assessments plays a key role for successful assessments QMOOD Quality model • We used the QMOOD quality model in MIDAS and found it to be satisfactory • MIDAS is generic: any appropriate quality model can be used Importance of principles/constraints • Feedback from projects shows using principles and constraints to report violations is practical as well as very useful MIDAS

38. Research Directions & Future Work Constraints reference model • A readily usable reference model of constraints is needed • We’ve already addressed lack of a readily usable model for design principles by creating our own; yet to do it for constraints • A reference model for design quality is needed • For product quality, ISO 9126 is a widely accepted reference model Plan from next FY • Deploy MIDAS assessments in a larger scale within CT DC AA Other future plans • Augment and categorize the current knowledge of design principles and project-specific constraints as a knowledge base • Evaluate and purchase relevant design analysis tools • Investigate how MIDAS can be integrated in Agile contexts MIDAS

39. Thank You! A Q MIDAS