NCEP Model Implementation Process

1. NCEP Model Implementation Process

2. Model Implementation Process The Basics • Plan • Test and Evaluate • Scientific • Technical • Customer Impact • Review • Implement

3. Model Implementation Process Develop Model Implementation Plan • Charter • Deliverables • Tasks • Schedule • List of Subjective Verification Participants

4. Develop Model Implementation Plan The Charter • Formally Recognizes the Existence of the Project • Objective • What You Intend to Accomplish • Background • Why It Needs to Be Done • Approach • How to Accomplish the Objective • Expected Duration • When Will the Project Be Done • Organizational Scope • Who Are the Stakeholders

5. Develop Model Implementation Plan Deliverables and Tasks • Deliverables • Product of a Task Being Completed • Some Deliverables may be Considered Milestones • Examples • Charter • Email Stating Test Results are Valid • Briefing Package for NCEP Director • Tasks • Actions Performed to Create a Deliverable • Steps Needed to Complete the Project

6. Develop Model Implementation Plan Roles and Responsibilities and Schedule • Roles and Responsibilities • Defines who is Responsible for Each Task • Same Person Responsible for Task and Deliverable • Schedule • Time Estimated for Each Task • Dependencies Among Tasks Considered • Level of Risk Assessed for Each Task • Schedule Created Based on Estimates, Dependencies and Risks

7. Develop Model Implementation Plan List of Subjective Verification Participants $Create Letter Asking for Participation in Evaluation $Describes Proposed Change $Asks for Intent to Participate in Subjective Verification $Defines Participant=s Responsibilities $List Defined By Response to Letter

8. Model Implementation Process The Basics • Plan • Test and Evaluate • Scientific • Technical • Customer Impact • Review • Implement

9. Model Implementation Process Perform Test and Evaluation • Scientific Test and Evaluation • Science Review • Technical Test and Evaluation • Customer Impact Evaluation

10. Perform Test and Evaluation Scientific Test and Evaluation • Conduct Retrospective Testing • Conduct Parallel Testing • Analyze Computational Resources • NCO Will Validate Analysis in Technical Test and Evaluation

11. Scientific Test and Evaluation Retrospective Testing $Low Resolution or Component Testing $Runs Made on Alternate Season from Parallel $Objective Verification Performed Against Runs

12. Scientific Test and Evaluation Parallel Testing $Proposed Implementation Configuration Run in Near Real Time $Typically Run on Development Resource $Output Made Available to Subjective Verification Participants $Subjective Verification Performed Against Runs $Objective Verification Performed Against Runs

13. Scientific Test and Evaluation Analyze Computational Resources • Model Timing Estimates • Model Data Storage Estimates • Product Format/Content Change Summary

14. Perform Test and Evaluation Science Review • $Panel Formed to Review Verification Results • $Subjective Verification Participants • $EMC Branch Chief • $NCO PMB Branch Chief • $OS&T Representative • $Telephone or VTC • $Reflects NCEP=s Commitment to Responsibly Review Quality of Proposed Implementations

15. Perform Test and Evaluation Technical Test and Evaluation • Validate Initial Test Setup • Analysis of Production Resources and Schedule • Timing and Scaling Tests • Compare Resources Needed Against Production Schedule • Run Parallel To Test Code Reliability • Analyze Data Requirements • IBM CCS Disk Use • Network Bandwidth and Downstream Servers • Analyze Product Content • Test Product Changes Against Core Software Display Systems

16. Perform Test and Evaluation Customer Impact Evaluation • $Impact Analysis Performed to Insure Proposed Implementation is Valid Business Case • $NCEP Chief Operating Officer • $OS&T Data Review Group (DRG) • $OCWWS Requirements POC • $ Telecommunications Operations Center (TOC) Director

17. Model Implementation Process The Basics • $Plan • $Test and Evaluate • $Scientific • $Technical • $Customer Impact • $Review • $Implement

18. Model Implementation Process Final Review • $Results of Scientific, Technical and Customer Impact Evaluations Packaged Into Briefing • $Briefing Presented for Approval • $EMC Director, NCEP Chief Information Officer and NCEP Chief Operations Officer • $NCEP Director • $Final Approval Must Be Received from NCEP Director Prior to Implementation

19. Model Implementation Process The Basics • Plan • Test and Evaluate • Scientific • Technical • Customer Impact • Review • Implement

20. Model Implementation Process Develop Implementation Package • $Submit JIF (Job Implementation Form) Package • $Create Installation Plan • $Create Notification Package • $JIF Memo • $Technical Procedure Bulletin (TPB) • $TOC Data Change Notification • OS&T Technical Information Notice (TIN)

21. Summary of the Model Implementation Process • Every Implementation Should Have Components of Planning, Test and Evaluation, Review and Implementation • The Level of Rigor Applied Depends On the Cost, Risk and Political Implications to the Organization • Simple Technical Implementations May Have Complex Scientific Implications • Simple or Non-Scientific Changes May Have Complex Customer Impact and Technical Implications