CS 577a FCR Feedback, Fall 2009

1. CS 577a FCR Feedback, Fall 2009 Winsor Brown, Barry Boehm, Supannika Koolmanojwong October 30, 2009 1

2. ARB Review Success Criteria • FCR • For at least one architecture, a system built to that architecture will: • Support the Ops Concept • Satisfy the Requirements • Be faithful to the Prototype • Be buildable within the budgets and schedules in the Plan • Show viable business case • Key stakeholders committed to support Foundations Phase (to DCR) • DCR • For the selected architecture, a system built to the architecture will: • Support the Ops Concept • Satisfy the Requirements • Be faithful to the Prototype • Be buildable within the budgets and schedules in the Plan • All major risks resolved or covered by risk management plan • Key stakeholders committed to support full life cycle 2

3. Overall FCR Feedback • Generally done well: presentations, client rapport • Reconcile FCR content with ARB Success Criteria • Define ALL key terms, acronyms in SID-glossary • If you’re deferring or skipping a normally-included artifact, explain why (e.g. COTS internals unavailable) and note in SID-document tailoring section. • Occassional order changes without telling us in a modified agenda • Role of primary DEN/remote student often mis-stated • IS System/Project Engineer (S/PE) • Includes IIV&V (also done by second DEN/remote student) • Includes Shaping (and re-shaping throughout semesters) 3

4. Overall FCR Feedback - 2 • When asked a question: • Give the answer in brief, this will help you in time management and the Review Board will get the desired information • Do NOT answer back while Review Board attempts to provide guidance • Many had poor time management; indicated that presentation(s) had NOT been practiced • Occassional pointing at laptop screen, not projected image • Very occassionally, slides with NO value added • At least one team used "template" from 577a 2008: implies an unethical process 4

5. OCD Feedback (1) • Generally done well: Organizational goals, Operational concept, System boundary and organizational environment. • Some benefits chains need rework: • Added stakeholders: users, clients, developers, IIV&Vers, database administrators, maintainers, interoperators, suppliers • Assumptions are about environment not about outcome • Involvement/use of system before system is built • System boundary diagram • If you are using the component in/for your system, remove it from environment, e.g. PHP, .NET framework. 5

6. OCD Feedback (2) • Organization Goals • Are Benefits Chain End Outcomes, • NOT project Initiative contributions • Identify Levels of Service properly • 100% availability, 100% reliability - not feasible! • Make sure you can measure LOS goals • Prototypes and System are NOT the same (usually) • Business Workflow • Use activity-type diagram • Illustrate business activities • Not technical/system activity • May not even “see” system explicitly u 6

7. Current BusinessWorkflowExample from 2008

8. Prototype Feedback • Generally done well: GUI Prototypes, Good understanding of client’s needs • Prototype all high-risk elements, not just GUI’s • COTS interoperability, performance, scalability • Use user/client-friendly terms • “John Doe, 22 Elm St.” not abstractions “Name1, Addr1” • Use as an opportunity to gather more information and/or examples • Identify end users and try to get feedback from end users • Focus on important and high priority requirements (initially) 8

9. SSRD Feedback • Generally done well: Project and Capability requirements, OCD-SSRD traceability • Prioritize all the requirements • Propagate LOS goals from OCD into SSRD or drop LOS requirements from SSRD (and SSAD) • Distinguish between client imposed requirements (SSRD) and developer choice solution (SSAD) • Make sure all requirements are testable • Qualify “24/7” Availability with noted exceptions • Update the new requirements in WikiWinWin tool • There is no such thing as an “implicit requirement” 9

10. No-SSRD Feedback • Result of a good after class question by a team doing NDI/NCS intensive but who found SSRD contents useful. • Why not requested? Because you can't impose "requirements" on COTS or NCS; alternatively, COTS/NCS capability become the requirements • Next year we will have NDI/NCS submit WinWin Report as part of packages • This year: put information in SID; they are needed for Test Case Development.

11. SSAD Feedback • Generally done well: Overall views • Follow UML conventions (arrows, annotations, etc.) • Generalization of actors • Uncommon mistakes in use-case diagrams • Two actors-one use case (means BOTH present) • Arrow direction for <extends> or <includes> • Devil is in the detail; simple is the best • Only two teams (3 and 14) had an adequate start on Information & Arctifacts Diagram • Read the exit criteria for the milestone carefully 11

12. Team 3 Information & Artifacts

13. Team 14 Information & Artifacts

14. LCP Feedback - 1 • Generally done well: overall strategy, roles and responsibilities • Fill in 577b TBDs • Identify required skills for NN new team member(s) (577b; if needed to meet "team size") • Show (concentrate on) your future plan; not the past • Full Foundations [nee Elaboration] phase plan • Don’t plan ONLY for documentation • Include Modeling • Include Prototyping; coding; executable architecture 14

15. LCP Feedback - 2 • COCOMO drivers • Usually differ per the module • PMATs rationale was often wrong: • CS577 projects' process maturity is between 2 and 3 • NOBODY did the detailed check list in the SwCEwCII book! • Some driver rationales were "ridiculous" • Add DEN Student interactions to the Gantt Chart • IIV&V • System/Project Engineer (includes Shaping) • Add maintainer’s responsibilities 15

16. FED Feedback • Generally done well: Business case framework, risk analysis • Specify LOS feasibility plans • Include training, operations, maintenance, opportunity costs/effort • Few had developers hours as cost • Try to quantify benefits, show return on investment • Change ROI to reflect on-going costs (possibly savings) • Distinguish one-time from annual costs in business case • Benefits start in mid 2010 (go to 6 month granularity);Costs start mid 2009 • Elaborate process rationale • Complete section 6 – COTS Analysis 16

17. SID Feedback • Requirements traceability • OC  WinC  SSRD  SSAD • Update every time there is a change • Update Glossary! • Glossary MUST have all system/project specific terms • Non-standard (unusual uses) 17

18. QFP • Generally done well • Some missing traceability injection-removal matrix • Some seemed to try to snow us with data, not present just a quick summary • Did NOT specify type of "Peer Review" • Pass around or Buddy Check (NOT desirable: no record of concerns) • Agile Artifact Review • Agile Internal/Informal (Role Based) Peer Review • Will have lots more to say at DCR! 18

19. Remote Students: System/Project Engineer (IIV&V & Shaping) • Generally done well • Constructive comments-how to improve • Some Shapers did not follow instructions: • No WinC summary (mostly numbers)[I am expecting update by email for those told] • Other parts incomplete or mis-understood 19

20. Things to improve • Presentation – communication skill • One word wrong could lead to billion $ loss. • Practice in front of others • Be concise and precise • Consistencies among each artifact • Team work vs. integrated individual works • Prepare your client: • Tell them what an ARB is (use agenda, success criteria) • Tell them what to expect from ARB • Time management • Get in and set-up ASAP • Have documents & client present 20