Sample Brown Paper Findings

1. Sample Brown Paper Findings Release Improvement Project

2. Understand the “As-Is” release processes Stay at the “forest” level Findings not process How things usually happen, not all the what-if loops Provide a medium to solicit process-user input Begin to build enthusiasm for the change process Brown Paper Objectives

3. Built six brown papers: C30 Release – SRC Operations D10 – Defect Repair D20 – Release Planning* Joint team building sessions with xxxxxx information holders: SRPM focused Multifunctional resources Held four brown paper faires to solicit user input from Developers, QA, PUBs, Product Management, etc. Brown Paper Approach • Findings incorporated into IPM Process panel set

4. Wide Spread TSG Support and Enthusiasm for the Brown Paper Process Is Exemplified by the Large Participation Brown Paper C30 D10 D20 SRC Operations Defect Repair SRPM Planning Participants At least 100 people provided hands-on input to the brown paper writing over 1,000 comments.

5. C30, D10, D20Findings

6. Shortfalls in the “process” are compensated for by great people-effort Release core team is a good concept Weekly submittal windows help drive quality into the system – social system allows people to work from same starting point Periodic Q&A sessions around software release have been helpful D20's upfront architecture work seems to have built a solid base so far Strengths

7. “Development is starving for senior management to impose rules, guidelines and well defined processes.” “Processes are complicated because all development organization don't follow the same requirements/guidelines.” There Is No One Set Process to Release Software C30 D10 D20 Releases differ by milestones, rules, and work steps

8. “I'm releasing in C30 as long as possible to avoid the D10 rules” “I didn't release in D10 because I would have had to create new Softdocs.” “D20 is consciously built to not look like D10.” To Utilize Multiple Release Vehicles Means Becoming an “Expert” for Each Release C30 “Release Expert” D10 “Release Expert” D20 “Release Expert” Development avoids multiple threads as much as possible.

9. “I bypass weekly test cycle with ‘under the table’ submits to QA.” “Too many rules have evolved. So I release with work arounds.” “If you are endorsed by the right people you can always get your code in.” “Techs network informally to get different components for a certain release.” Beating the System Becomes the Norm

10. “…Code freeze often turns to code slushy.” TPR freezing seems arbitrary depending on release: Freezing too early can cause noncritical problems to never get fixed Nothing Is Every Truly “Frozen” • What we think: • Freeze content • Freeze functionality • Freeze SEV 0 + 1 TPRs • Freeze schedules • What we do: • Unfreeze content • Unfreeze functionality • Inflate TPR severity to SEV 3 and repair • Slip deadlines Slushy

11. Numerous Softdoc iterations/compressed CD Rom Production PUBs rework/short-manual lead times “Content slush leads to document rewritings” SUT rebuilds/refreshes Limited Schedule Credibility Has Down-Stream Impact Slushy

12. “QA tests should be run on development systems by developers.” “Too much integration testing in QA.” “We need division level integration testing processes.” “Development provides QA with unit test plans and QA provides Development with system test plans… Differentiation is unclear.” “Need visibility of QA's ability and readiness to perform testing; need criteria and metrics for start of QA.” Blind Hand-Offs from Development to QA Lead to Rework Weekly Development/QA Magtape Frisbee Toss

13. “Windows were first introduced as a bandaid…not meant to become part of the process.” “Developers chase Windows more than they pay attention to quality.” “With Window system, developers are out of business at least 28% of the time.” “Weekly SUTs are cut, no matter what, to hit the schedule; we shouldn't cut SUTs if code doesn't run or can't be installed.” “We need to stop sacrificing quality for schedule.” “It's hard to fix, build, test, and submit code in a week.” “Windows should be controlled on the SRC side: Development could submit anytime and SRC could coordinate the code.” Weekly Submittal Process Is Characterized as Inefficient and Schedule Driven

14. “Why request quarterly input when releases are less frequent than quarterly.” “No rewards or penalties for making or missing schedules.” “Shouldn't commit to plan until firm design plans are in place.” “Program and schedule plans were severely handicapped by lack of QA staff early on the program.” Implementers Have Limited Input to the Planning Process Resulting in Noncredible Schedules… Decision Makers Release Plan Implementers PUBs QA Product Management Developer Manager Developer SRPM …But are responsible for deliverables.

15. “Core teams lack consistent behavior, rules… they need training.” “Release content is being determined by non-technical people.” “Core team is too far removed to determine what should and shouldn't be allowed for submittal.” Core Team Roles and Responsibilities Are Unclear

16. 23 weeks of iterations…still, not reached “one-good-SUT” milestone Different opinions exist for TPR filings “Need to marry old and new school thought” “We never slow down enough to get to functional milestones.” D20 Seems Stuck in a Vicious Cycle “I can't build a good product without a good SUT” “I can't give you a good SUT until you give me a good product” Development SRC/SRPM

17. SRC Findings

18. People - "SRC Staff are always willing to help" - "SRC people exhibit a strong desire to work through a difficult process" BIT testing procedure - "BIT testing process is excellent at finding "dead on arrival" SUTs" - "Frequent submittal windows are helpful" Strengths

19. Complex Release Procedures and Rules Cause Confusion and Frustration Much energy is expended on low value added activities- statusing , manual validation

20. Lack of Flexibility in Release Rules Results in an Inabilityto Distinguish Between Major and Minor Problems Typos in SOFTDOCS are treated the same as code failure

21. Limited Screening/Testing by Development Leads to Errors Being Caught Later in the Release Cycle Costs to rectify errors increase the further downstream in the process . . . SRC is the developers "watchdog"

22. Antiquated Tools Enslave Rather Than Empower the SRC Process Most release tools are at least 5 years old and no longer support s xxxxxx's current and future needs

23. Too Many SOFTDOCS "Gods" Lead to Confusing Rules No one "owns" SOFTDOCS. As a result: • There is a lack of standardization and documentation • Users use SOFTDOCS to fulfill a variety of information needs: •• Dependency identification •• Internal documentation •• Installation Procedures SOFTDOCS serves many masters . . . none very well

24. Release Planning Findings

25. “At least we're planning now. We used to not do it.” “Market requirements and product objectives process is designed to prioritize market requirements and translate to objectives. The goal is good.” “Customer Advisory Boards are a great idea. Let's do more. Let's involve customers in our future.” “Quarterly updates to development plans are helpful. I refer to this a lot.” “The strategy briefings are a great improvement in communication.” “For what does get into the formal process (commitments), we do well.” “Planning for software tied to hardware is better than for other software.” “…xxxxxx's top 10 planning priorities are useful.” “The iterative steps in the process provide ‘good visibility’ to the plan.” Strengths of Release Planning

26. “Most of this (customer input) is missing. We are problem driven.” “All xxxxxx groups must return their focus to customer needs — not the political BS of internal affairs.” “Need more direct interaction with customers.” Customer Input Is Only Weakly Tied to Release Planning “Customer Needs” “xxxxxx Needs” The planning process is largely internal, not customer driven.

27. “Customers need change. The process doesn't address this well.” “We need a process to update resource plans, not just steal resources.” “We don't do enough to see if we funded a project properly. We borrowed resources from other projects.” “Development resources are often diverted from projects, making schedule estimating/committing difficult.” Planning Doesn't Adjust to Mid-Year Changes Well Meeting schedules is difficult when resources are “stolen” mid-project.

28. “The second year of a three year plan is often changed, and the third year is never right.” “The last project I was on, the three year plan was off by 2 1/2 years.” “Three year plan numbers are meaningless for a lot of projects—made up. Evaluating cost/schedules from three year plans isn't basing designs on reality.” “My three year lasted six months.” Many Employees Feel the Three Year Planning Process Is Ineffective Planning beyond one year has little value to employees.

29. “We don't' teach project management, so people don't know how to make accurate plans.” “We lack both skills and historical data to estimate accurately.” “We don't know how to estimate, so we don't know how to make schedules or meet them, so plans are meaningless.” “We never do post-mortems on project budgets.” Development Planning Is Ineffective Planning capabilities need to be developed.

30. “Release bottleneck: New product or enhancement is complete but can't be shipped except as an IPM due to all or nothing Guardian releases.” “Customers won't wait forever. Plan for frequent, incremental improvement.” “The world changes more frequently than once a quarter.” “Release process cycle time (too long) leads to changes in assumptions, which leads to development rework to “fit” with new assumptions.” Frequently of Release Is Red Issue for Development Products are waiting at the station, but the train isn't ready to board.

31. “We need to set up an information directory…” for plans product dependencies customer commitments “…and make it available to all.” Too many private distribution spreadsheets and documents with limited distribution “Need to send automatic reminders to person assigned…” “FREs go to Product Management. Once that happens, they are never heard from again.” Release Process Information Is Not Shared SNAX depends on spooler SNAX has no dependencies Required information is not readily available.

32. “Release rules are not consistent and change frequently. We need to know the rules.” “Decision criteria are not understood, justified, and communicated.” Role of core team and middle/executive management questioned: “Who are these people?” “They do not understand…” Release Rules and Decisions Are Unclear …made by a few, but impact many.

33. “QA and Development should coordinate their plans before submitting to release planning.” “xxxxxx should have a Master Plan, from which the Release plan can be derived.” “Publications plan not well integrated within release plan.” “Publications plan not included in 3-year Development plan.” Functional Plans Are Not Integrated What's the plan? My Plan My Plan Release plan does not drive all work activity.

34. “Only about 30% of the commitments get into the process.” “Don't appropriately differentiate between major projects and little stuff.” “Who analyzes customer commitment requests? Product Management doesn't, and Development is isolated from customers.” “Establish and stick to priority commitments. Don't tweak so frequently.” “There are still many customer commitments which do not get viability. Commitments are made without enough thought.” Communication of Priorities and Commitments Is Inadequate I'll write down that commitment tomorrow We'll have that to you next Tuesday