NAWIPS Developer Training

1. NAWIPS Developer Training Scott Jacobs Steve Schotz NCEP/Central Operations Systems Integration Branch 18-19 April 2006

2. Overall Topics • Introductions and Schedule • History / Background • Development Process • Coding Standards and Common Errors • Library Guide and Header Files • Tables and Help, PDF and Error Files • GEMPAK Fundamentals • Test Programs and Other Common Debug Tools • Utilities and Access to SIB Systems • Upcoming and Long-term projects NAWIPS Developer Training

3. Schedule • 4 sessions • Day 1 • 9:00-12:00 • 1:30-4:30 • Day 2 • 8:30-11:30 • 1:00-3:30 • 1 or 2 breaks during each session • Ask questions! NAWIPS Developer Training

4. Day 1 - Morning • Introductions • History / Background • Development Principles • Development Process NAWIPS Developer Training

5. Introductions • Name • Where do you work? • What do you do? • Why are you here? NAWIPS Developer Training

6. Topics • N-AWIPS mission • N-AWIPS history • N-AWIPS customers • Overview of N-AWIPS functionality • N-AWIPS development principles • Overview of N-AWIPS development process NAWIPS Developer Training

7. N-AWIPS Mission Develop robust meteorological workstation software to meet NCEP requirements, NCEP-AWIPS (NAWIPS) • Provide access to high resolution/global datasets • Provide flexible meteorological analysis and display tools • Provide integrated product generation capabilities for current and future modernized products • Provide robust, centrally supported and maintainable software NAWIPS Developer Training

8. N-AWIPS History • GEMPAK- Developed at NASA during 1980s to support research and academic communities (foundation for N-AWIPS) • Provides set of ~ 150 libraries, (reusable code) for decoding, data basing, performing calculations, displaying, and product formatting • Extensive use in university and research communities • Exchange of ideas, algorithms and codes • GEMPAK migrated to NMC- Early 1990s • National Centers (NC) given responsibility to address NC specific requirements – Late 1980s • Recognition of significant mission differences between WFOs and NCs • Decision primarily cost driven NAWIPS Developer Training

9. N-AWIPS History • Transition Project (TP) formed in Office of Director - 1991 • TP mission - address NCs workstation modernization requirements • AWIPS contract awarded – 1992 • TP begins deployment of NTRANS into operations – 1993 • TP folded into NCO – 1995 • AWIPS operational – 1999 • New AWIPS contract awarded to Raytheon – 2005 NAWIPS Developer Training

10. N-AWIPS Customers • NCEP service centers • Climate Prediction Center; Hydrometeorological Prediction Center; Ocean Prediction Center (Camp Springs, Maryland) • Aviation Weather Center (Kansas City, Missouri) • Storm Prediction Center (Norman, Oklahoma) • Tropical Prediction Center (Miami, Florida) • NCEP supercomputer and Environmental Modeling Center • NWS Alaska & Pacific regions • NWS River forecast centers • Unidata (~200 universities + private industry) • Government labs (MDL, ARL, ...) NAWIPS Developer Training

11. High Level Requirements • Diverse products, geographic and temporal scales • Somewhat diverse forecast processes • Mostly common/generic software requirements NAWIPS Developer Training

12. N-AWIPS Fundamental Capabilities • Data visualization • Integrated product generation NAWIPS Developer Training

13. N-AWIPS Data Visualization Capabilities • Definition: • Ingest, decoding, calculation and display of meteorological data • Meteorological data types include: • Radar and satellite imagery • Model data: grids, MOS • Observations: METAR, marine, aircraft, upper-air, lightning, pilot reports, profilers, … • Forecast products: flash flood, Sigmets, Airmets, watches, warnings, …. NAWIPS Developer Training

14. N-AWIPS Data Visualization Capabilities • Critical requirements include: • Flexible user-defined geographic and temporal scales and map projections • Wide variety of user-specified, “on the fly” calculations • Grid Diagnostics ~170 built in grid operators that allow user specified combinations of calculations, e.g., adv((vor(geo),thrm(hght@300:700)) • Variety of display options, e.g., station plotting models, contour attributes, etc. • User-defined data overlay options • Key display functions: • Multiple animation loops: max of 16 loops with total of 120 frames • Automatic data time matching to frame time to support data overlay comparisons • Procedures for automatic loading of data into multiple loops • Continuous roaming to allow detailed views of data over large geographic regions NAWIPS Developer Training

15. Integrated Product Generation • Definition: • Creation/Editing of graphical products overlaid on meteorological data displays • Critical requirements: • Drawing and editing tools using meteorological objects, e.g., fronts, weather symbols, etc. • Graphical objects navigated to account for multiple product sectors and projections, wind rotation, … • Facility to import first guess fields, e.g., model fields and other centers’ graphical products • Product formatting to support GIF, TIFF, PostScript, GRIB, BUFR, Redbook… • Product layering to support multi-component or multi-time concurrent editing • Object grouping NAWIPS Developer Training

16. N-AWIPS Development Principles • Maximize Code Maintainability • Software must be written to be understood/maintained/enhanced by someone other than original developer(s) • Must be operationally robust • Maximize Code Reuse • Employ functional libraries with test programs • Never write code more than once to perform a particular function • Maximize Functional Flexibility/Adaptability • Generic functionality to meet broad spectrum of current and anticipated needs, e.g., flexible calculation tools, display options • Maximize Hardware Independence - Unix (IRIX, SUN, HPUX, AIX), Linux, single or multi-headed workstations • Multiple hardware platform support improves software robustness • Enables user platform/configuration choices • Employ industry standards, e.g., ANSI C, no extensions NAWIPS Developer Training

17. N-AWIPS Development Principles • Minimize dependence on commercial external software systems • Keep the system as simple as possible • Facilitate installation and testing • Reduce user costs of N-AWIPS NAWIPS Developer Training

18. N-AWIPS Development Process Overview • Use an evolutionary development strategy • Build in small frequent increments, quarterly release cycle • Refine requirements based upon forecaster feedback • Continue to improve system to support new products, refinements of existing products and increase forecaster productivity NAWIPS Developer Training

19. N-AWIPS Development Process Overview • Employ disciplined development process covering entire software development life cycle • Repeatable process • Process must be efficient to allow changes to the system to be cost effective • Traceable process and document trail with peer reviewed check points NAWIPS Developer Training

20. N-AWIPS Development Process Road Map NAWIPS Developer Training

21. Development Process Topics • Requirements Analysis • Proposal • Detailed Design • Coding and Developer Testing • Independent Testing • Release Notes • Approvals NAWIPS Developer Training

22. Requirements Analysis • Attempt to understand the user needs • Must ask a lot of questions to make sure the needs are well understood • Don’t assume anything • Think about what the users are saying • Ask the users to explain anything that is not clear • Don’t skip this step • It always takes longer to fix something than it would have to learn the requirements first • Even for small tasks, spend some time on requirements NAWIPS Developer Training

23. Proposals and Designs • Purpose is several fold • Reduce costly downstream errors • Creates a repeatable process • Documents the approach and design for future reference • Ties the requirement to the coding and testing NAWIPS Developer Training

24. Proposals • Task Goal • 1-2 sentence statement of what you want to accomplish • Identify the requirement number addressed • Background • Explain user requirements • Review current functionality • The length of the proposal is dependent on the complexity of the task NAWIPS Developer Training

25. Proposals • Proposed Solution or Options • Describe any algorithms • If there is more than one solution, list them for review and discussion • Also include any pros/cons to each option • Areas of Code Affected • Identify the libraries or applications • E.g., All GD programs; Only GDPLOT2; DG library; PGEN NAWIPS Developer Training

26. Proposals • Specific Tasks • If this will be a multi-part task, list the subtasks • For example • Part 1: Create new library routines and add to test program • Part 2: Incorporate new routines in applications • Each subtask should be able to be tested alone • Test Plan • Create a preliminary test plan • Will help you make sure that you have covered everything • This will also help the reviewer(s) NAWIPS Developer Training

27. Proposals See example document • Proposals are reviewed by • Lead Software Engineer (LSE) • NAWIPS Team Lead (NTL) • Other designers, as needed • In some cases, users or their representatives • Discussion • If needed, a meeting will be arranged to discuss various aspects of the proposal • Approval • A proposal is approved when either the LSE or the NTL initials the document • Approval may be contingent upon changes indicated in the body of the document NAWIPS Developer Training

28. Detailed Design • Description • Overview of the changes • Summarize the proposal • Length is dependent on the complexity of the task • Designs are also done for bug fixes • In this case reference the bug or problem report NAWIPS Developer Training

29. Detailed Design • Function List • Include any calling sequences changes (CSC) • Include the entire sequence with the changes • If no CSC, only indicate “No CSC” • Describe the changes needed for each function • Include enough detail so that someone other than you could do the coding • Also include any functions to be deleted NAWIPS Developer Training

30. Detailed Design • Library Check • This is important when changing GEMLIB/CGEMLIB functions • Many applications call these library functions • Changing a library routine for one task may have unintended consequences in some other application • This check attempts to find all locations where a particular function is called in order to assess the impact • This is also needed to ensure that all necessary testing is included in the test plan • The utility “whocalls” can be used to help fill this in • Utilities will be discussed more fully later NAWIPS Developer Training

31. Detailed Design • Header Files, Tables, etc. • List any files, other than source code, that also need to be modified or created • Test Plan • Include all tests that must be run • Test the intended changes • Test any applications found from the Library Check to ensure no change in functionality NAWIPS Developer Training

32. Detailed Design • Author • Make sure to put your name in as the author • Update the Level of Effort • This should be a number of days to finish the coding, unit testing and independent testing • Update the start and end dates • These dates are for when you start and finish editing the detailed design • If the design is returned without approval, and you modify the document, indicate that as “Version 2” with the new dates NAWIPS Developer Training

33. Detailed Design See example document • Designs are reviewed by • Lead Software Engineer (LSE) • Others as needed • NAWIPS Team Lead (NTL) • Other designers • Discussion • If needed, a meeting will be arranged to discuss various aspects of the design • Approval • A design is approved when the LSE initials the document • Approval may be contingent upon changes indicated in the body of the document NAWIPS Developer Training

34. Coding and Developer Testing • Copy existing files or create new files in your local account • Do not work in the software account • Make sure you get all necessary tables, etc. • Also get the link scripts • Compile scripts are all in $GEMPAK/build • Link scripts are in the directories with the application source code • Be careful with file names if working on GPLT functions since the same name is used in APPL and GPLT; and GPLTDEV and DEVICE NAWIPS Developer Training

35. Coding and Developer Testing • Proceed with the development as outlined in the detailed design • If you find that the design is incomplete or incorrect, update the design and make a note in the document header • If you are not the original author of the design, discuss it with that person first • Make sure that you update the log in each source file • Compile on as many platforms as you can • There should be no compiler errors nor warnings when you are ready to deliver • Access to SIB systems will be covered later NAWIPS Developer Training

36. Coding and Developer Testing • Conduct the developer unit tests • Test the changes as outlined in the test plan • Collect any special data sets for later testing • Test unusual situations • That is try to break the code with strange input • This is easiest if the new routines have been added to the appropriate test program NAWIPS Developer Training

37. Coding and Developer Testing • Delivery • Create a directory tree for your changes that mimic the software account • Start at the “gempak” directory • Put all files in the appropriate subdirectories • Enter your name and the dates you started and finished coding on the form • Enter the path to the delivery tree you created • Access to SIB systems will be discussed later • Describe the changes in a few sentences • This description will become the log entry in “nawips.log” NAWIPS Developer Training

38. Coding and Developer Testing • Delivery (cont.) • List all modified files • Put those with calling sequence changes first • List all new files • List all files to be deleted • All files listed will be compiled by the configuration manager (CM) • If a header file is changed, make sure to indicate all source files with dependencies on that header file • These files will also be compiled by the CM • List all applications that must be linked NAWIPS Developer Training

39. Coding and Developer Testing • Delivery (cont.) • Describe the tests conducted and any significant results • Indicate any platform specific information • Provide a path to a set of executables for all platforms • This is for the code reviewer • Update the testing section of the Design with the following • Describe any special data sets and their location • Provide any special testing instructions not covered in the original design test plan NAWIPS Developer Training

40. Coding and Developer Testing See example document • Code is reviewed by either • The task owner • This is the person who wrote the design, if someone else did the coding • Another lead designer • Lead Software Engineer (LSE) • Approval • A delivery is approved when the reviewer initials the document • It is then given to the CM to add to the baseline software account • The CM enters the delivery number on the form NAWIPS Developer Training

41. Independent Testing • Conducted by someone not already associated with the task • Must test on all platforms • List all tests done • Must do all tests in the test plan • Anything else the person can think of to break it • List any problems encountered • These would indicate a serious condition and would need to be addressed as soon as possible • Also list any observations and suggestions • These are less serious, and may be addressed later NAWIPS Developer Training

42. Independent Testing See example document • The task owner reviews the test report • Writes responses to all problems • If there are serious problems, the task goes back to coding • Writes responses to all observations and suggestions • If any warrant immediate fixes, the task goes back to coding • If there are no issues or delayed issues, the task is DONE! • Approval • The task owner initials and dates the test report after review NAWIPS Developer Training

43. Release Testing • Test scenarios to emulate operations • List of commands and button clicks • Product creation tests • Modified as needed to accommodate different data sets • Batch scripts • NTRANS metafile generation • New scripts to test all programs • Allow comparison between versions • Can and should be run at various times throughout the release cycle NAWIPS Developer Training

44. Release Notes See example document • When are Release Notes required • The user interface changes • Results or output change • Public calling sequences change • At the discretion of the NTL • Should be written as early as possible in the development process NAWIPS Developer Training

45. Release Notes • Should sufficiently describe the changes to functionality • Ranges from a few sentences to a few paragraphs • Will be incorporated into the full Release Notes document and the “What’s New” help file in NMAP2 • The NTL reviews and edits the release notes for consistency with the design NAWIPS Developer Training

46. Day 1 - Afternoon • Coding Standards • Common Errors • Library Guide NAWIPS Developer Training

47. Coding Standards • Principles • Make the system extensible so that additional needs can be met • Don’t need to serve all needs at once • Isolate complexity as much as possible • Code must be able to be maintained by someone other than the original developer • Code should be readable and consistent • Functions should be as short as possible • Utilize functional libraries to maximize code reuse • Routines/functions should be independently testable • Use of test programs will be discussed later NAWIPS Developer Training

48. Coding Standards • Fortran [77] • All files should consist of only one subroutine or function • Library routines should be well defined with a single purpose • Lines should be less than 72 characters • Do not use the GOTO statement NAWIPS Developer Training

49. Coding Standards • Fortran [77] • Routine Names • GEMLIB routines are 2 character library name + underscore + 4 characters to describe routine • Program routines are 2 character program class + 1 character program id + 3 characters to describe routine • GPLT routines all start with “G” and have up to 6 characters • DEVICE routines all start with “D” and have up to 6 characters • Driver routines all start with “H” and have up to 6 characters NAWIPS Developer Training

50. Coding Standards • Fortran [77] • Uppercase • All Fortran keywords • Subroutine and function names • Parameters defined by the PARAMETER statement • This is for readability since Fortran is case independent • Variables • Lower case • Should not exceed 6 characters • Implicit typing • I-N are Integers, only • All others are Reals • All Characters must be declared NAWIPS Developer Training