1 / 21

Toward standardization of an automated software quality model: the Grid-QCM

This study introduces Grid-QCM, an automated model for certifying software quality in research projects, referring to QA concepts and strategies. It offers a structured approach to quality assurance in software development.

gennyj
Download Presentation

Toward standardization of an automated software quality model: the Grid-QCM

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toward standardization of an automated software quality model: the Grid-QCM Adriano Rippa Engineering Ingegneria Informatica S.p.A. ETICS 2nd EC Review (CERN) 15 February 2008

  2. Summary • Introduction to QA concepts • The starting point of the study • The proposed Grid Quality Certification Model (Grid-QCM) • FAQs • Timeline and feedbacks • Conclusions ETICS 2nd EU Review - CERN - 15 February 2008

  3. Quality Assurance refers to several concepts • Quality of the implementation process • High level steps of the software production cycle suggesting what the organization must do (not how) to have effective development processes that “may lead” to good software. • Quality of the requirements management • Correct collection/management of requirements and relation with the customer and stakeholders, to reduce the percentage of failures due to misinterpreted requirements. • Quality of the service • Performances and correctness of the service • Finally quality of the software… QUALITY Quality of Product Quality of Process CMM Grid-QCM ITIL ISO – 900x ETICS 2nd EU Review - CERN - 15 February 2008

  4. Some QA Terminology • According to ISO 9126 documentation we can define: • Measure: the number or category assigned to an attribute of an entity by making a measurement (sometimes used as synonymous of metric) • Metric: The defined method to measure an attribute and the scale • Measurement: The use of a metric to assign a value (which may be a number or category) from a scale to an attribute of an entity) ETICS 2nd EU Review - CERN - 15 February 2008

  5. Quality is a matter of measure! • State-of-the-art provide hundreds of metrics • The most common • Cyclomatic complexity • Lines of Code • Function Points • Mean Time Between(to) failure • Bugs density • … • Other approaches (Goal Question Metric - GQM) promote user defined metrics • Anomalies distribution • Effort used to solve anomalies • Cost of not founded anomalies • … ETICS 2nd EU Review - CERN - 15 February 2008

  6. Quality Standards • There are many standards to asses the quality of the processes of an organization • CMM • ISO family • ITIL • AQAP • But QA means initial investments and managing QA means devote resources to it! • Several studies show that lot of companies (e.g. many Small/Medium Enterprises) can’t afford the initial effort and don’t recognise the promised increase of value. • Only ~70 companies in the world are certified at level 5 • 50 of them are in India • Only 25% of the companies in the world are level 2 or above ETICS 2nd EU Review - CERN - 15 February 2008

  7. QA in research project is missing! • Current quality assurance standards are useful but • They need lot of time to be applied. The organisation need to be structured and certified. What for the short-live consortia? • They provide only theoretical guidelines which need to be adapted and realised. What for the objectivity? • It’s hard to systematically verify goodness of results: managing tools needed • They need resources to be devoted to • People need training and certification needs inspections and time to be achieved ETICS 2nd EU Review - CERN - 15 February 2008

  8. QA in grids: oursolution • Grid-QCM is amodelforquality assurance that is • fully automatable in measuring and verifying activities to reduce investments and management effort, • not subjective, to certify the object not the process nor the organization, • product oriented, not process oriented, • …easily adoptable within (Grid) Research Projects ETICS 2nd EU Review - CERN - 15 February 2008

  9. Grid-QCM: Preliminary Remarks (1/2) • Grid-QCM is independent from the ETICS tools: it just needs “automation” . • Grid-QCM has been developed within a Grid project and to asses the quality of grid software research projects but it can be used for any software applications . • Grid-QCM has been developed according to • The gotten feedbacks from expert people and potential users • several standards: • Grid-QCM has been described according to ISO standards (e.g. ISO/IEC 25000, 14598) • Grid-QCM has been restructured according to ISO 25041 • Quality attributes has been named using the same terminology as ISO 9126 ETICS 2nd EU Review - CERN - 15 February 2008

  10. Allows Code analysis Allows automation Defines metrics Runs measures The ETICS vision Grid-QCM ETICS SW (v.2.0) ETICS grid infrastructure CERN, INFN, UoW (NMI) ETICS 2nd EU Review - CERN - 15 February 2008

  11. Grid-QCM: Structure • Grid-QCM is structured in Evaluation Modules (EM). • The set of evaluation techniques are grouped in families. Every family is an EM • 5 EMs: • EM: Static analysis • EM: Coding style • EM: Structural testing • EM: Functional testing • EM: Standards compliance ETICS 2nd EU Review - CERN - 15 February 2008

  12. Evaluation Modules 1/2 • EM Static analysis • Quality characteristics: • Reliability – maturity • Maintainability – analysability • Maintainability – changeability • Maintainability – testability • Static analysis of classes. Statistics on measures are used as predictor of quality characteristics. • EM Coding style • Quality characteristics: • Maintainability – analysability • Static analysis of the source code. ETICS 2nd EU Review - CERN - 15 February 2008

  13. Evaluation Modules 2/2 • EM Structural testing • Quality characteristics: • Functionality – accuracy • Reliability – maturity • Structural testing to classes identified more likely to have many errors. • EM Functional testing • Quality characteristics: • Functionality – accuracy • Functionality – interoperability • Reliability – maturity • Portability – adaptability • Portability - installability • Platform compliance and to functional abilities of the software • EM Standards compliance • Quality characteristics: • Functionality – standards compliance ETICS 2nd EU Review - CERN - 15 February 2008

  14. Grid-QCM: Final Score • Final score should be provided according to the following schema. The items which should be available for the users are: • A table summarizing the results • A list of passed and non passed tests • All the important information as: • Software product (e.g. name, version, executable code, documentation..) • Platform (name, version, date) • Quality characteristics (name, evaluation result, evaluation module identification) • Standard compliance (for each standard: name, version, date) • Identification of evaluation report (organization, report number, date) • Identification of certification body (organization, contact information) • Certification data (dates, certification number) • Electronic signature of certification record ETICS 2nd EU Review - CERN - 15 February 2008

  15. Grid-QCM: Summarizing Table example G = Good M = Medium P = Poor ETICS 2nd EU Review - CERN - 15 February 2008

  16. Grid-QCM: FAQ • I can’t add any overhead to my project • This model (and the capability of automate) reduce the effort in performing continuous build and test activities (e.g. coverage tests) on different SW releases. • How much costs adopting it? • Nothing, the model is free and licensed under Apache 2.0. • ETICS is provided as a service running on a dedicated infrastructure, free of charge for Research Project. • My organisation is certified ISO/CMMI so I… • The model complement and complete the ISO/CMMI certification rules. It can be a standalone model or a tool to be compliant against ISO/CMMI standards. • ETICS is just one tool as requested from certification rules. ETICS 2nd EU Review - CERN - 15 February 2008

  17. Current ETICS metrics and Grid-QCM ETICS 2nd EU Review - CERN - 15 February 2008

  18. The timeline EGEE ’07 1-5 Oct Budapest (Hun) OGF 21 15-19 Oct Seattle (USA) ESA 3rd GRID & e-Collaboration Workshop 16-17 Jan 2008 Frascati (IT) OGF 20/EGEE UF 7-11 May Manchester (UK) Now EELA 3° Conference 3-5 Dec 2007 Catania (IT) OCTOBER DECEMBER FEBRUARY MAY QUALIPSO Conference 16-17 Jan 2008 Rome (IT) ECHOGRID/EUChinagrid Conference 24-25 April Beijing (CHINA) Belief Conference 25-28 June Rio de Janeiro (BRA) ETICS 2nd EU Review - CERN - 15 February 2008

  19. Grid-QCM: feedbacks • The model has got good feedbacks and acknowledgements • People are asking for more information • Some issues about the name of the model (solved) • One issue about ISO9000 compliance (outside the scope?) • Requests for metrics about the process  outside the scope? ETICS 2nd EU Review - CERN - 15 February 2008

  20. Conclusions • Grid-QCM is a certification model • Automatable • Free • Not in contrast with classical standards • Ready to interact with classical standards • Not limited to the ETICS build and test tool • Not limited to research projects • Not limited to grid software • Grid-QCM require less human effort to be used because it is almost fully automatable • ETICS tool is ready to implement Grid-QCM • During the ETICS 2 project, Grid-QCM will be proposed for standardisation under ISO. ETICS 2nd EU Review - CERN - 15 February 2008

  21. Q&A http://www.eu-etics.org • Acknowledgements: • Many thanks for his contribution to Andrea Manieri ETICS 2nd EU Review - CERN - 15 February 2008

More Related