170 likes | 188 Views
Strategic Activities of Global Acceptance on Software Testing. Dec. 6, 2010. Gum-Ho Choe Accreditation of Software Testing Working Group, APLAC Technical Committee. Contents. ·. Ⅰ . Background Ⅱ . Activities on Software Testing Ⅲ . Supplementary Requirements for SW testing
E N D
Strategic Activities of Global Acceptance on Software Testing Dec. 6, 2010 Gum-Ho Choe Accreditation of Software Testing Working Group, APLAC Technical Committee
Contents · Ⅰ. Background Ⅱ. Activities on Software Testing Ⅲ. Supplementary Requirements for SW testing Ⅳ. Future Work
Background • An Accreditation of Software Testing has emerged as a key • points to improve product competiveness. • *Necessary to improve the product and process such as smart phone, etc. • Automotive Electrical/Electronic System (AEES) additionally • requires the Security Management System (SMS) 1/9
Activities on Software Testing Classification for Assessment for SW testing • To verify the conformity of product, process, • and system (quality & security) Certification Conformance Testing Product Process Conformity Assessment ISO/IEC 9126 12119 ISO/IEC 15408 15504 (SPICE) CMMI System ISO 9001 TL 9000 26262 FDIS (Security)
Supplementary Requirements for SW Testing in KOLAS Scope • KOLAS-SR-015 is applicable to accredit testing labs. • which conduct Testing related to security, embedded, • bio-recognition, game, and mobile software, etc. . References • ISO/IEC 17025, ISO/IEC 9126-1, -2, ISO/IEC 25001, • ISO/IEC 25020, ISO/IEC 25030, ISO/IEC 25051, • ISO/IEC 25062 • For specific test for Software, IEC 61508-3, • ISO/IEC 24709-1, -2, etc.
Summary of KOLAS-SR-015 Assessment Checklist • Assessment Checklist for Documentation • - Product Manual : Basic information, Functionality, Reliability, • Usability, Efficiency, Maintainability, Portability • - Document for Users : Integrity, Accuracy for Explanation, • Consistency, Understanding, Outline • (2) Assessment Checklist for Quality Properties • - Specified based on Quality model for external and internal quality, • ISO/IEC 9126-1 : Functionality, Reliability, Usability, Efficiency, • Maintainability, Portability • (3) Assessment Checklist for Accreditation of Testing Lab. • - Specified based on the Checklist of ISO/IEC 17025
Overview of KOLAS-SR-015 Structure • The structure of KOLAS-SR-015 follows ISO/IEC 17025 • Structure of KOLAS-SR-015 • 5. Technical Requirements • 5.1 General • 5.2 Personnel • 5.3 Accommodation and environmental conditions • 5.4 Test methods and method validation • 5.5 Equipment • 5.6 Measurement traceability • 5.7 Sampling • 5.8 Handling of test items • 5.9 Test reports
Content of KOLAS-SR-015 5.1 General • This document describes supplementary requirements • for accreditation in software field • This document is based on ISO/IEC 17025.
Content of KOLAS-SR-015 5.2 Personnel • The technical manager shall have the following • knowledge, education & training and the ability to • monitor and order properly. • ① testing activities including preparation for test • reports • ② knowledge of software • ③ knowledge of software test and evaluation • Testing labs shall have at least five testing • personnel including one technical manager
Content of KOLAS-SR-015 5.3 Accommodation and Environmental conditions • Testing labs shall have the policy and procedure to • guarantee that the confidential information and • customers’ ownership are protected. • Transmission path shall be secure in order to • protect the confidential test information. • Testing Labs shall give the minimal right to access • the testing room
Content of KOLAS-SR-015 5.4 Test Methods and method validation • Test labs shall use test methods in international • standards as follows • - ISO/IEC 9126-2 • - ISO/IEC 25051:2006 • Test labs shall perform the risk-based analysis • through reviewing the test cases. • Defect severity, status, and so on shall be managed, • and defect handling procedure shall be established. • For software field, test labs shall not apply • estimation of uncertainty of measurement
Content of KOLAS-SR-015 5.5 Equipment • Equipment may include the software and other • machines used in evaluation and tests • Test labs have manuals that describe how to operate • all equipment. 5.6 Measurement traceability • Measurement traceability shall not be applied in • software tests • * Test equipment -- Validation
Content of KOLAS-SR-015 5.7 Sampling • Sampling in software tests refers to the selection of • test cases • The examples of sampling are the following: • (1) the selection of the test cases to test the • combinations of different conditions and variables • (2) the selection of the re-execution test cases for • regression tests
Content of KOLAS-SR-015 5.8 Handling of test items • The configuration management of customer's • submissions and test documents shall be performed 5.9 Test reports • Test results shall be reported to a technical manager.
KOLAS-Accredited Labs for SW Current Status • KOLAS has accredited 7 testing laboratories for SW Accredited Laboratories
KOLAS-Accredited Labs for SW Accredited Laboratories
Future Work Focusing on Product Quality and Security • Based on CC, ISO/IEC 9126, 25000, and IEC 61508, etc Extending to Security Management System • Extending to ISO 26262 FIDS for SMS in AEES * Ready to certify the SMS by CBs such as SGS-TUV, UL, etc