1 / 20

Estimating Document Peer Review Effort

Estimating Document Peer Review Effort. Richard D. Stutzke (SAIC), Terry L. Palmer (Davidson Technologies, Inc.), and William Beckett (U.S. Army Aviation and Missile Command, Research, Development and Engineering Center, Software Engineering Directorate)* 25 October 2001

lobo
Download Presentation

Estimating Document Peer Review Effort

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Estimating Document Peer Review Effort Richard D. Stutzke (SAIC), Terry L. Palmer (Davidson Technologies, Inc.), and William Beckett (U.S. Army Aviation and Missile Command, Research, Development and Engineering Center, Software Engineering Directorate)* 25 October 2001 Presented at the 16th International Forum on COCOMO and Software Cost Modeling, Los Angeles, 23-26 October 2001. *Point of Contact: Richard.D.Stutzke@saic.com or 256-971-6624 Approved for public release. Distribution is unlimited. Est PR Effort (17Sept01)

  2. Topics • Background • Model Formulation • Data Analysis • Results Est PR Effort (17Sept01)

  3. Basic Equation Effort = (Amount/Productivity)*(# participants) where Effort = total effort for team [phrs] Amount = size [pages] Productivity = scan rate [pages/(phr*person)] Est PR Effort (17Sept01)

  4. Peer Review Activities • Author organizes and distributes materials • Reviewers prepare (individually) • Review meeting (consolidate comments) • Author corrects document (rework) • Moderator reviews and approves • Participants • Author • Reviewer • Moderator • Scribe • Presenter) ( The Review Process Est PR Effort (17Sept01)

  5. Scope of Effort Estimation Model* Est PR Effort (17Sept01)

  6. Effort Estimation Model* PR Effort = NP*(NR/PrepRate + NT/MtgRate)* EAF Where = NP*EAF*[NR/Prep Rate + (NR +1)/Mtg Rate] NP = # Pages NR = # Reviewers NT = # in Team = NR + 1 (Reviewers + Author) PrepRate = Preparation Rate [# pages/(person*phr)] MtgRate = Meeting Rate [# pages/(person*phr)] EAF = Effort Adjustment Factor *Assumption: both Moderator and Scribe prepare too. Est PR Effort (17Sept01)

  7. Data Characteristics • Documents Used • New development only (all pages are new) • Written in Microsoft Word • No re-inspections • Summary • 68 documents considered • 18 documents selected for analysis • Size range: 7 to 124 pages Est PR Effort (17Sept01)

  8. Reported versus Actual Pages (All Reviews) 140 120 100 80 y = 0.9493x Reported Pages 2 R = 0.5199 60 40 20 0 0 20 40 60 80 100 120 140 Actual Pages Est PR Effort (17Sept01)

  9. 140 120 y = 1.0197x 2 R = 0.9835 100 80 Actual Pages 60 40 20 0 0 20 40 60 80 100 120 140 Reported Pages Reported versus Actual Pages (Selected Data) Est PR Effort (17Sept01)

  10. 30000 25000 y = 180.32x 2 R = 0.8714 20000 Actual Words 15000 10000 5000 0 0 20 40 60 80 100 120 140 Actual Pages Word Per Page Est PR Effort (17Sept01)

  11. 7.000 6.000 5.000 4.000 Prep Time (min/[person* page]) -0.6257 y = 13.654x 2 R = 0.3617 3.000 2.000 1.000 0.000 0.0 20.0 40.0 60.0 80.0 100.0 120.0 140.0 Doc Size (actual pages) Prep Time versus Doc Size Est PR Effort (17Sept01)

  12. 6.000 5.000 4.000 Time (min/[person*page]) -0.9298 y = 30.787x 2 R = 0.6515 3.000 2.000 1.000 0.000 0.0 20.0 40.0 60.0 80.0 100.0 120.0 140.0 Doc Size (actual pages) Meeting Time versus Size Est PR Effort (17Sept01)

  13. 6 5 4 # of Peer Reviews 3 2 1 0 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 3.75 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 6.00 6.25 6.75 7.00 More Time Ratio Distribution of Time Ratios Est PR Effort (17Sept01)

  14. Prod Coeff (All People, Best PRs) 9.0 8.0 7.0 6.0 -0.7673 y = 37.331x 2 R = 0.7055 5.0 Effort (min/pg* person) 4.0 3.0 2.0 1.0 0.0 0 20 40 60 80 100 120 Size (actual pages) Est PR Effort (17Sept01)

  15. 9 8 7 6  = 0.19  = 0.21 5 Count 4 3 2 1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 More Defects/page Document Quality Distribution Est PR Effort (17Sept01)

  16. 7 6 5  = 2.44  = 2.03 4 Count 3 2 1 0 1 2 3 4 5 6 7 8 More Cost (phrs/defect) Total Cost per Defect Est PR Effort (17Sept01)

  17. Effort Adjustment Factor • Document complexity and “readability” • Text (“readability”, “vocabulary size”) • Tables (# rows, # columns) • Figures (?) • Structure (# outline levels) • Reader Ability and Knowledge • Capability (ACAP, PCAP, language proficiency, cultural bias) • General knowledge (AEXP in domain, PREC?) • Specific knowledge of product (PREC? UNFM) • Process (checklists, structured reading) Est PR Effort (17Sept01)

  18. Some Possible Measures of Document Complexity Est PR Effort (17Sept01)

  19. Mosenthal and Kirsch Readability Metric Est PR Effort (17Sept01)

  20. Recommendations • Save a machine-readable copy of the document that was actually reviewed • Use automated tool to count and record the size and characteristics of the document • Record effort to nearest 0.1 phrs (0.5 phr too gross) • Do separate reviews for • Sections written by different authors • Source code and documents • Subdivide large documents Est PR Effort (17Sept01)

More Related