220 likes | 356 Views
Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study. Andrea Manieri Adriano Rippa - Engineering Ingegneria Informatica s.p.a. On behalf of ETICS Project. The starting point. Current quality assurance standards are useful but: are process oriented
E N D
Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study Andrea Manieri Adriano Rippa - Engineering Ingegneria Informatica s.p.a. On behalf of ETICS Project
The starting point • Current quality assurance standards are useful but: • are process oriented • provide theoretical guidelines on the organization • Theproblem: • Guidelines can be applied but it’s hard to systematically verify goodness of results • Oursolution: • A quality assurance model • fully automatable in measuring and verifying activities • not subjective ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
The timeline OGF 21 15-19 Oct Now ALL HANDS MEETING 23-25 May OGF 20/EGEE UF 7-11 May FIRST BETA VERSION FINAL BETA VERSION FIRST OFFICIAL VERSION FEBRUARY MAY OCTOBER DECEMBER ECHOGRID/EUChinagrid Conf. Beijing 24-25 April QUALIPSO Conference Dec 2007 Belief Conference RIO – BRASIL 25-28 June ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Terminology • Measure: a value (number or category) assigned to an attribute of an entity (e.g. software component) • Measurement: The act or process of assigning a number or category to an entity to describe an attribute of that entity. • Metric: A function whose inputs are software measures and whose output is a single numerical value that can be interpreted as the degree to which software possesses a given attribute that affects its quality. • Perspective: A set of homogenous software attributes that lead to some quality assumptions ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Gird Quality Assurance Certification Model (GQACM ): a 3D QA Model • Let’s suppose A, B, C to be different perspectives • Most of Common QA Standards (e.g. CMMI) are Waterfall-like • GQACM uses 3 independent perspectives to provide client-friendly results • Users can be interested in some attributes and less in others.. • So-made perspectives can have sublevels… A B C A C B ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
GQACM: Preliminary remarks • GQACM is independent from the ETICS tools • However, any GQACM implementation must use automation – the most useful characteristic of the model is that it can be automatically applied! • GQACM has three perspectives (I.e. independent points of view) to derive the quality degree of a software : • Code and test • Platforms • Standards ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Perspectives composition • Code Test Perspective: to qualify the goodness of the code and of the test • Coding Conventions • Code Commenting • Other QA metrics related to the corresponding programming language • Unit and functional tests • Platform perspective: to qualify B&T requirements on different platforms • The set of GQACM available platforms • Standard perspective: to qualify the compliance with standards (e.g. IPV6, BES, JSDL, SRM, WS-I) • The set of GQACM available standards ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Test execution metrics • Cyclomatic complexity • CCT (method level): • if C is the cyclomatic complexity (of the method) and T is the number of executed tests then CCT = C/T • RFCT (class level): • if CK RFC is the the number of methods which can be invoked from an instance of the class target to answer a message ) and T is the number of executed tests on it then RFCT = RFC/T • ET (class level): • if E is the number of events which interact with the class and T is the number of executed tests then ET = E/T • NOCT (class level): • if CK NOC is the Number Of Children of a class and T is the number of executed tests on the class then NOCT = NOC/T ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Code comment metrics • FC: • if F is the number of functions and C is the number of comment paragraphs then FC = F/C • AD: • Automatical documentation programs use: e.g. doxygen ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Coding convention metrics 1/2 • Only one class definition per header file • Filename = Classname • Filename in lowercase • No cycles in “include” operations • No C instruction (in C++ programming) • Coherent Programming Convention • Curly Brackets positioning • First character in variables names • Lower case in macros names • Coherent use of new lines • Semicolons only at the end of a line ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Coding convention metrics 2/2 • No Never used functions • No Not reachable code paths • No Not defined prototypes • No Never used constants • No Never used macros • No Never used types definitions • No Never used variables • Too frequent “breaks” use • No absence of variables de-alloaction • No Null pointers • Management of one instruction long “if” or one instruction long “cycle” • Shared resources management ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
other metrics • Other software related metrics • (e.g. Chidamber and Kemerer (CK) Metrics for OO software) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Final Score1/2 • For every perspective must be genrated: • a score between 0 and 100 • a list of passed and not passed tests • The way to calculate the score is different for every perspective. • For code perspective • The score is simply calculated using this mathematical formula: (X/N)*100 where X is the number of successes, N the number of used metrics ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Final Score 2/2 • For platform perspective • If • DP is the number of declared compliant platforms, • CP the number of really compliant platforms, • TP the number of GQACM available platforms , • CTP the total number of compliant platforms , • then the score can be calculated with this mathematical formula: (CP/DP)*50+(CTP/TP)*50 • For standard perspective • If • DS is the number of declared compliant standards, • CS is the number of really compliant standards, • TS is the number of available standards, • CTS is the total number of compliant standards (declared + not declared), • then score can be calculated with this mathematical formula: (CS/DS)*50+(CTS/TS)*50 ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Score visualization • The graphical view of the score is related to the used test tool • In general, all tools have to use at least the “classical version”: • Final score: <total>/<max possible score> ([<perspective><score>]) • Where • Perspective can be: Code (C), Platforms (P), Standards (S) • Score: 0-100 (where 0 is 0% compliant, 100 is 100% compliant) • Total score: sum of the scores • E.g. component X: GQACM score = 151/300 (C90-P51-S10) • ETICS could complement the standard basic visualization using a 3d graph to give the user a better feeling interface ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
GQACM integration in an ISO 9126 certified organization • Facts: • ISO 9126 define quality attributes . • GQACM support the measure and evaluation of quality metrics. • Hypothesis: • ISO9126 Portability: adaptability, installability • ISO9126 asks OS adaptability and installability for software of certified organizations. • GQACM may test deployment of SW on different platforms (HW+OS) • ISO9126 Maintainability: all. • ISO9126 asks to develop a software that is easy to analyse, easy to modify, Stable • GQACM metrics (such as the CK for OO or coding convetions metrics) helps to verify in concrete these requirements ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
GQACM integration in an ISO 9126 certified organization • ISO9126 Functionality: accuracy. • ISO9126 asks that a software have to give results according with requirements. • GQACM certification can run functional test (or plug-in external test suits) of declared compliances. • ISO9126 Functionality: adequacy. • ISO9126 asks for correct functions with specific goals. • GQACM may check not dead functions. • Our thesis: • Many ISO9126 quality metrics can be measured and evaluated based on GQACM • Any tool implementing it and can be integrated as supporting ISO9126 adoption ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
GQACM integration in a CMMi certified organization • Facts • CMM is process oriented (it gives guidelines). • GQACM is product oriented. • Hypotesis: • CMM2 • asks to “control and value” • GQACM propose the measures and metrics to be collected, hence controlling and evaluating specific sw related items • CMM3 • asks to “use internal standards and have a reference model” • GQACM proposes code and implementation conventions (how to organise and structure code, how to define tests, etc.) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
GQACM integration in a CMM certified organization • CMM4 • asks to “have measured and controlled processes using quantitative and statistical techniques” • GQACM propose the continuous collection of quality data to perform trends analysis. • CMM5 • asks to have an “improvement process and quality based on measurement” • GQACM provide numerical reference to set the improvement processes!! • Our Thesis • GQACM is not in contrast with CMMI and can be integrated in organization as a tool to support CMMI adoption ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
ETICS B&T tool and GQACM • ETICS: • Could perform many measurements and calculate metrics (e.g. JPmdPlugin and JFindbugsPlugin) Code and test perspective • Can build and test software on several distinct platforms Platform perspective • Could test the software compliance with standards Standard perspective • Can automate all the processes (build, deployment, test) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Conclusions • GQACM is a certification model • Automatable • 3D • Not in contrast with classical standards • Free • GQACM doesn’t require big efforts to be used ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007
Thanks Questions? Feedbacks? http://www.eu-etics.org ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007