1 / 41

Improving Testbed Identification, Creation, Check-out and manipulation.

Improving Testbed Identification, Creation, Check-out and manipulation. Global Initiatives Team – June 12, 2006. Topics. Project Team Team Charter - Business Case, Problem Statement, Scope DMAIC Define – SIPOC and Critical to Quality (CTQ) Measure – Data Collected Appendix.

ellery
Download Presentation

Improving Testbed Identification, Creation, Check-out and manipulation.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Testbed Identification, Creation, Check-out and manipulation. Global Initiatives Team – June 12, 2006

  2. Topics • Project Team • Team Charter - Business Case, Problem Statement, Scope • DMAIC • Define – SIPOC and Critical to Quality (CTQ) • Measure – Data Collected • Appendix

  3. Project Team Add Hoc Team HTS: Emilo Sanfalice TBC (TBD) TBC (TBD ChampionGreg Geppert Process Owner Kim Simmons LeadChris Columbia Core Team PPD: Chris Whyde Mary Gerred Alla Crowder Lata Walsh Tony Rumilla Toby Brooks Scott Brubaker Ci Hye Kim P&M: Pat Fabian Dave Jarvis BPG Richard Mitchum

  4. Team Charter - Business Case • It is important that the testbeds for the release be developed in the shortest time frame and at the lowest cost possible. the customer wants and needs prior to project scheduling, task planning, and resource allocation. • Assumptions, misinterpretations, changes in customer wants and needs after scheduling, result in increased costs (rework), risks (schedule and quality), and negative affects on morale (unscheduled overtime, perceived wasted effort, shifting priorities and violation of the function-quality-schedule relationship).

  5. Team Charter- Problem Statement • With the addition of MortgageServ testing in November to the quarterly release, the need to identify, select and protect test cases loaded to the test region for use as testbeds has become a critical piece of the preparation. • Since August 2005 excluding STAR WARS, 27 projects required a total of 82 PCR’s, of which 39 (47%) were associated with business requirements and 18 (22%) resulted in functionality that was not delivered with the release. The STAR WARS project included 94 PCR’s of which 27% were associated with business requirements. • For projects delivered in 2006, more than half of the HTS customer survey responses indicate that the time planned to complete the BRD was not appropriate (56%) and that the BRD did not contain the necessary detail to develop the system according to what the customer (end user) expected to see in the final product. (61%).

  6. Over 1800 testbeds were created for the May release. The test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. • The CS and Coll Cross functional teams actually stopped testing for periods of time to assist GL and Ins Svc. In testbed preparation. Over 40% of the testbeds provided to GL had to be reworked due to lack of detail in the initial request.Over 40% of the testbed request from Ins Svc required additional investigation by the QD team before that could be completed.

  7. Team Charter - Project Scope In Out Effectively leveraging analysis and requirements documentation from IR and the initiation phase All aspects of the Requirements and analysis phase including Business, PMO, Compliance, QD and HTS practices and participation as it relates to the BRD development PCR and Requirements quality measures Initiation Phase and IR development PCR creation and change management after BRD Functional and technical specifications Test execution practices Testing defect management Measuring employee satisfaction with the BRD process

  8. DMAIC Define March 23 (End) Measure April 19 (End) Analyze May 5 (End) May 15 (Solutions -End) October 31 (Implementation -End) Improve November 1 (Begin) Control September 1

  9. DMAIC – Define (SIPOC)

  10. DMAIC - Define (Critical to Quality) Level 2 Requirement Level 1 Requirement Output/Need Accurate- includes end user needs Quality Scope – clear definition of desired vs. must Easy to Understand Business process flow-what it will look like BRD Draft Time allocation for research & analysis Consulting- right amount of time for analysis Creation – right amount of time to create Collaboration – decisions by consensus Key resources Commitment and participation (available and active) The right HTS SME’s are involved Knowledge – systems and business – assumptions validated Primary Customer: HTS

  11. DMAIC - Define (Critical to Quality) Level 2 Requirement Level 1 Requirement Output/Need Compliant –All compliance requirements included Quality Completeness – nothing missing Accurate – without errors Thoroughness- no impact to other areas overlooked BRD Draft Easy to Understand Visual – flow charts at right level of detail Plain English – written at right level of language Concise – No irrelevant information Efficiency Turn around time Cost – no re-work of same analysis or reviews Primary Customer: Business Cost – no HTS scope or cost changes

  12. DMAIC - Measure What our Customers Told Us (HTS) More than 61% are not satisfied The BRD provided HTS all details needed to develop the system according to how the end user expected to see it in the final product More than 55% are not satisfied The correct amount of time was planned and used for the BRD development avoiding follow-up meetings and requirements changes after BRD sign off.

  13. DMAIC - Measure What our Customers Told Us (QD) All business requirements, including compliance and legal requirements have been included in the BRD avoiding additional requirements gathering meetings and changes to the requirements after final sign-off. Business tells us that they have higher levels of satisfaction than QD After the BRD sign-off there were no HTS scope or cost increases due to missing or unclear requirements in the BRD.

  14. DMAIC - Measure PCR Analysis Almost 50% of PCR’s on projects delivered in 2006 were associated with BRD quality The larger the project the more likely we see PCR’s

  15. DMAIC - Measure BRD Risk and Milestone Performance Most of the risk is associated with larger sized projects, however it does not appear to show any significant correlation project size Analysis: The PPD database contains 127 projects of which 20 were in yellow and/or red status (16%) prior to BRD completion. No Correlation To Project Size

  16. DMAIC - Measure BRD Revisions and IR Inputs A correlation was not found between use of an IR and number of BRD revisions We have seen 35% improvement in use of IR’s

  17. DMAIC – Analyze Y = f(x) Micro Problem Statement 1Y= Why do we not have the correct SME and decision makers in the BRD process (lack of proper knowledge and skills?)X= Business, QD, and HTS expectations for the BRD have not been developed or approved (no common understanding of what is expected to be delivered) X= No established process to identify SME's(Tasks and timeframes, training, reference materials, Communication plan, roles and responsibilities) Micro Problem Statement 2Y= Why do we have lack of commitment from required SME's and decision makers in the BRD process?X = Lack of a planning process with established roles and responsibilities to define and validate the WBS and BRD and establish a accurate resource and time estimate X = SME resource availability is not validated Micro Problem Statement 3 Y = Why do 38% of the HTS customers and 18% of the QD customers tell us that they are not satisfied with how easy the BRD is to understand while only 4% of the business is unsatisfied?X = Business, QD, and HTS expectations for the BRD have not been developed or approved

  18. DMAIC – AnalyzeY = f(x) Micro Problem Statement 4Y = Why are 56% of HTS customers unsatisfied with scheduled time for BRD consulting and creation while only 4% of the business is unsatisfied?X = Measurements are not available to identify the cost (budget, cycle time, quality, morale) associated with failure to follow the scheduling processMeasurements needed to move from a culture that relies on a History of making it happen and MBO goals tied to unplanned dependencies on new functionalityX = Lack of a planning process with established roles and responsibilities to define and validate the WBS and BRD and establish a accurate resource and time estimateX = SME resource availability is not validated Micro Problem Statement 5 Y = Why do 61% of the HTS customer survey responses and 18% of the QD responses tell us that the BRD does not meet their expectations for necessary detail, while only 9% of the business responses feel we have not met their expectation? X =Business, QD, and HTS expectations for the BRD have not been developed or approvedX= Lack of measures to show impact of BRD quality

  19. DMAIC – ImproveProcessSolution 1:Establish BRD Standards • Define the level of detail and format expected from each customer group (HTS, QD and Business). • Project criteria used to allow expectations to be defined for projects which include new product and product changes as well as those which have minimum impact on workflow. • Implement a process to “tailor” and clearly establish and validate the BRD expectations at project kick off.

  20. DMAIC – ImproveProcess Solution 2:SME Validation Process • Skills and time necessary to complete BRD tasks clearly defined and validated with SME and SME management. Confirmation of resource availability at kick off. • Process established to update and maintain departmental resource inventory and capacity across projects. • Management and control process which includes re-validation, reporting, and escalation related resource commitments. • Establish a formalized process to track missed requirements to disengaged resources (impact and repercussions associated with commitment failures)

  21. DMAIC – Improve ProcessSolution 3:BRD Planning Process • Establish a framework for consistent planning all BRD tasks. Includes guidelines, templates and task definitions. • Planning framework uses project criteria (size, complexity, user group impact etc.) to establish standard lists of tasks and the expected time for completion. • Common and repeated task definitions are defined with objectives, roles and responsibilities and entrance criteria. • The appropriate level of iterative development is planned.

  22. DMAIC – ImproveMeasurement Solution:BRD Quality Scorecard • Utilize Customer Satisfaction Survey questions to establish baseline measures of satisfaction with Business Requirements CTQ’s for HTS, QD and Business. • Create a BRD traceability quality measure derived from an objective assessment of gaps between the Initial Request, Business Requirements and Functional Specifications. • Establish a process for using clear operational definitions of PCR root cause to establish measures of “good” and “bad” PCRs.

  23. DMAIC – Improve Leveraging Best Practices Process Improvement Best Practices

  24. DMAIC – Improve Goals • 25% reduction from baseline in unscheduled PCRs associated with business requirements defects (gaps, scope, inaccuracies, misinterpretations). • Achieve an increase of xx% in BRD quality (as measured by the functional specification traceability measurement) • Improve Customer Satisfaction Scores for HTS, QD and the Business: • Achieve a release level aggregate HTS customer satisfaction rating of 3 that the BRD provided all the necessary details to develop the system according to how the end user expected to see the final product and that the correct amount of time was used to develop the BRD (12% improvement) • Achieve a release level aggregate QD customer satisfaction rating of 2.5 for BRD quality (completeness, accuracy, consensus and ease of understanding) and BRD efficiency (turn around time and cost of re-work) (10% improvement) • Achieve a release level aggregate Business customer satisfaction rating of 2.1 for BRD quality (completeness, accuracy, consensus and ease of understanding) and BRD efficiency (turn around time and cost of re-work) (17 % improvement)

  25. DMAIC – Improve Implementation Timelines * See appendix for solution step details

  26. DMAIC – Control

  27. DMAIC – Control BRD Quality Dashboard

  28. Appendix • Process Flow Chart • Data Collection Plan • Fish Bone Root Cause • Scoring Solutions • Create BRD Standards Solution Steps • SME Validation Solution Steps • Project Plan

  29. Appendix DMAIC- Define (Process Flow)

  30. Appendix DMAIC- Define (Process Flow)

  31. Appendix DMAIC - Measure Data Collection Plan (Quality)

  32. Appendix DMAIC - Measure Data Collection Plan (Easy to Understand)

  33. Appendix DMAIC - Measure Data Collection Plan (Efficiency)

  34. Appendix DMAIC - Measure Data Collection Plan (Best Practices)

  35. Appendix DMAIC - Analysis (Fishbone Root Cause)

  36. Appendix DMAIC - Analysis (Fishbone Root Cause)

  37. Appendix DMAIC - Improve Scoring Solutions –Must/Want Criteria

  38. Appendix DMAIC - Improve Scoring Solutions –Must/Want Criteria Smaller numbers have greater importance

  39. Appendix DMAIC - Improve Create BRD Standards (Steps) Step 1 • Define the level of detail and format expected for each customer group (HTS, QD, and Business. • Implement a BRD template which meets these expectations within a single BRD. • Project criteria will be established to allow expectations to be defined for projects which include new product and product changes as well as those which have minimum impact on workflow. Step 2 • Implement a process to tailor the BRD expectations at project kick off • Create a process that uses project criteria to “tailor” and clearly establish the BRD expectations at project kick off. • It involves establishing a communication and agreement process with appropriate documentation related to expectations to ensure that the process remains both flexible and consistent..

  40. Appendix DMAIC - Improve SME Validation (Steps) • Step 1 • Prior to kick off meeting implement a process to identify the proper resource needs and gain commitment to this understanding • Step 2 • Create a process to confirm resource availability at kickoff (Both SME manager and the SME) Note: This confirmation process will expand to include the improved estimates developed from implementation of Step 4. • Step 3 • Create a process to update and maintain an inventory of resources by department and departmental resource capacity across projects • Step 4 • Prior to Kick off implement a process to include pre-kick off estimate of time required and when, define roles and responsibilities before assignment of SME, clear definition of skills needed and validation of correct SME • Step 5 • Create a management and control process which includes re-validation, communication, and escalation related to resource commitments • Step 6 • Establish repercussions for lack of commitment- formalized process to track missed requirements to disengaged resources

  41. Appendix DMAIC - Improve Project Plan

More Related