COQUALMO and Orthogonal Defect Classification(ODC)

COQUALMO and Orthogonal Defect Classification(ODC) Keun Lee ( keunlee@sunset.usc.edu) & Sunita Chulani (Sunita_Chulani@us.ibm.com)

COQUALMO and Orthogonal Defect Classification(ODC) • Current COQUALMO Model • - Results and Challenges • COQUALMO – ODC Research Approach • Example • Results • Issues and Research Plans

Current COQUALMO System COCOMO II Software development effort, cost and schedule estimate COQUALMO Software Size Estimate Defect Introduction Model Software platform, Project, product and personnel attributes Number of residual defects Defect density per unit of size Defect Removal Model Defect removal profile levels Automation, Reviews, Testing

Partion of COQUALMO Rating Scale COCOMO II p.263

COQUALMO Defect Removal Estimates - Nominal Defect Introduction Rates Delivered Defects / KSLOC Composite Defect Removal Rating

Multiplicative Defect Removal Model - Example : Code Defects; High Ratings • Analysis : 0.7 of defects remaining • Reviews : 0.4 of defects remaining • Testing : 0.31 of defects remaining • Together : (0.7)(0.4)(0.31) = 0.09 of defects remaining • How valid is this? • All catch same defects : 0.31 of defects remaining • Mostly catch different defects : ~0.01 of defects remaining

Example UMD-USC CeBASE Data Comparisons • “Under specified conditions, …” • Peer reviews are more effective than functional testing for faults of omission and incorrect specification(UMD, USC) • Functional testing is more effective than reviews for faults concerning numerical approximations and control flow(UMD,USC) • Both are about equally effective for results concerning typos, algorithms, and incorrect logic(UMD,USC)

ODC Data Attractive for Extending COQUALMO - IBM Results (Chillarege, 1996)

COQUALMO Extension Research Approach • Extend COQUALMO to cover major ODC categories • Collaborate with industry ODC users • IBM, Motorola underway • Two more sources being explored • Obtain first-land experience on USC digital library projects • Completed IBM ODC training • Initial front-end data collection and analysis

Digital Library Analysis to Date - in ODC terms • Artifacts • - Operational Concept, Requirements, Software Architecture documents • Activities • - Perspective-based Fagan inspections • Triggers • - Environment or condition that causes defect

Front End (Information Development) Triggers • Clarity – confusing or difficulty to understand information. • Style – inappropriate or difficulty to understand the manner of expression • Accuracy – incorrect information • Task Orientation - inappropriate presentation to perform task • Organization – relationship between parts is not conveyed • Completeness – missing information. • Consistency – the expression manner is not displayed in a consist manner

Initial Digital Library Project ODC Analysis - Trigger percentage Distribution by Team

Initial Digital Library Project ODC Analysis - Number of Triggers Defects by Team

Issues and Research Plans • Understand anomalies in Digital Library Data • Number of Team 22 defects • Team 4 completeness defects • Due to differences in artifacts or procedures? • Continue Digital Library ODC collection & analysis • - Detailed Design, code, test • Obtain, analyze industry ODC data • - Looking for more sources of ODC Data

COQUALMO and Orthogonal Defect Classification(ODC)