Successful Program Execution

1. Successful Program Execution Center for Systems and Software Engineering (CSSE) Executive Workshop 15 March 2006 Marilee J. Wheaton General Manager Systems Engineering Division ©2006 by The Aerospace Corporation

2. Key Attributes of Successful Program Execution • People and teamwork • Controlling risk • Processes • Accountability • Cost and schedule estimating • Stability • Industrial base

3. People and Teamwork • Contractor execution team and PM should be first line of defense for mission success • Second line is contractor functional organization check and balance • Third line is government, FFRDC and SETA team • Government is not staffed to be the first line of defense • Space Quality Improvement Council (SQIC)

4. What Is The SQIC? • Space Quality Improvement Council • An industry forum with government sponsorship • Identifies and addresses common problems and solutions • Executive-level engineering and quality control leadership from the NSS contractor community • Mission assurance processes • Product quality issues that affect all (Example: FPGAs) • Acquisition policy affecting quality • Facilitated by The Aerospace Corporation • Carries forward consensus industry recommendations to government sponsors • Not for attribution

5. Space Quality Improvement Council (SQIC)Key Accomplishments • Contractor data sharing • New SQIC/GIDEP NSS Advisory Forum, operational Sept. 2005 • Anomaly/mission failure investigations • Standardized MOU developed and ready when needed • Selective specs and standards reintroduction • Industry representatives on all standards committees • Industry consensus on specs and standards to put on contract • SMC and NRO requiring that these be on new contracts • Common playing field in bidding executable contracts • Key element in assuring mission success • Helps government be a “smarter buyer”

6. Smart Buyer 1 • Helping System Program Directors (SPDs) and acquisition executives become smarter NSS buyers • Be aware of the demands and expectations placed on your industry counterparts Smart Buyer 2 • Now that the buyer understands the contractor environment… The buyer must formulate an executable plan and maintain an executable baseline.

7. How Do We Control Risk? • Four variables in project management: cost, schedule, performance, and risk • Need to define risk rigorously and cap it at an acceptable level • If you cap the other three variables, risk grows continuously • Example: 1990s NASA Mars missions, which failed Assertion: Capping risk and driving schedule hard leads to best cost performance

8. How Do You Cap Risk? • Use standards for engineering and manufacturing • Appropriate test program, robust QA, PMP, etc. • Rigorous system engineering and test • Example: JPL Mars Rover “incompressible test list” • Test like you fly, fly like you test • Manage risk at a sufficiently senior level on a program • Allowing lower levels to trade mission success for cost and schedule leads to unbounded risk

9. How Do You Cap Risk? (Cont.) • Government/industry team manages risk incrementally • Robust mission assurance tailored to risk assessment • Contractor uses “buildup” process in design and test to define and manage risk • Establish confidence at each major step—not just at the end of the program • Find and fix defects early • Establish environment that encourages problem reporting • Example: Weekly Watchlist (“No Ambushes”)

10. Processes • Use validated processes that ensure predictable, repeatable results • Processes represent lessons learned and best practices from past experiences • Help prevent repeating mistakes • Rigorously manage the baseline implementation • Fund and manage disciplined, validated technology insertion • Require credible independent assessments prior to program initiation and at key milestones

11. Processes (Cont.) • Use standards and measure performance against those standards • Put core specs and standards on contract • Examples: PM&P, SE, test • Government program office and FFRDCs ensure that “best practices” are being used • Independent flightworthiness and mission assurance certification • Rigorous EELV mission assurance process has 2000 verification tasks prioritized against risk assessment • Underpins “thumbs up” in Flight Readiness Review (FRR)

12. Accountability • Nearly every failure in space systems resulted from a breakdown in accountability • Need clear definition of accountability on government/contractor team • At every organization level and flows down to individuals executing program • Key program people remain in the job long enough to ensure they are accountable for their decisions that affect mission success

13. Accountability (Cont.) • Key roles of government • Define requirements • Set risk level and assure responsible risk management • Ensure that proper practices are used • Incentivize contractor team and assess performance • Contractor balances holistic enterprise perspective and tailored program needs • Line organizations are accountable for the product • Functional organizations are accountable for the processes • Prime takes accountability for the quality of output by subcontractors and vendors • Streamline government-industry decision-making process:In-plant representation

14. Cost and Schedule Estimating • Recognizes that best value is not necessarily lowest cost bid • Government must place value on non-deliverables essential to mission success (Examples: SE, MA, QA,…) • Then industry will also value them • Exclude “name-that-tune-in-three-notes” bids • Has a well-established Independent Cost Estimating (ICE) and program control function • Budget program to 80% confidence, including a management reserve sized by risk • Expend reserves to execute unforeseen elements of baseline program—not new requirements

15. Stability • Stable, manageable baselines—requirements, budget, and schedule • Manage necessary but unplanned changes • Rigorous systems engineering process for assessing impact of new requirements • New requirements must come with new funding • Allows trade spaces vs. “cast-in-concrete” requirements • Capabilities, cost, and schedule • Architectures that allow right-sized programs (can be executed in about 5 years) • Regulates appetite of user community

16. Industrial Base • Compete when in the best interests of the government • Government should recognize the need for stability, longevity, and performance of industrial base • Government aligns contract and fee structure to balance risk between government and contractor • Example: Titan IV contract • A world-class supplier base • Multiple sources, particularly at sub-vendor level • Government should require and enforce subcontract management plan • Put it on contract (How do we evaluate and enforce?)

17. Knowing When We Are Turning the Corner Attributes of World-Class Industry/Government Acquisition Team • Effective programs that meet user demands with 100% mission success • Efficient execution—meeting cost, schedule, and performance baselines • Retirement of program risk as a function of time is measurable and visible • Government and contractor teams meet their commitments Promises Made = Promises Kept