NCAT Working Group Charter

1. Net Centric Analysis Tool (NCAT)Network Centric Operations Industry Consortium (NCOIC) Stockholm International Technical and Business Strategy MeetingJune 8-9, 2005 Hans Polzer(Lockheed Martin)NCAT Working Grouphans.w.polzer@lmco.comWG Chair – Fred Mervine (IBM)Contributors: Vice Chair - Vish Dixit (Boeing), Kevin Berthoud (Lockheed Martin), Kevin Shaw (BAE), Abe Meilich, PhD (Lockheed Martin)

2. NCAT Working Group Charter • To create an assessment mechanism that provides a metric based approach to evaluating/measuring a system’s, subsystem’s, or component’s “fitness” for operating in a net centric environment. • Work has focused on providing a prototype release of the NCAT for feedback from the ASA FT • Goal was to provide overview of this release here today • Presentation today will focus on several topics related to NCAT effort • Development of the NCAT • How to read and use tool • Plan for feedback from ASA FT membership

3. Related/Foundation Works • Net Centric Operation Warfare Reference Model (NCOW RM) • Office of the Assistant Secretary of Defense for Networks and Information Integration/Department of Defense Chief Information Officer (OASD/NII DoD CIO) • The purpose of the NCOW RM is to set forth a common, enterprise-level reference (i.e., a single, authoritative Department source) for the development of DoD Enterprise Architectures and for acquisition programs to use in focusing and gaining net-centric support within the Global Information Grid (GIG) • Net Centric Checklist • Net-Centric Checklist May 12, 2004 Version 2.1.3 (publicly available on the web) • http://www.defenselink.mil/nii/org/cio/doc/NetCentric_Checklist_v2-1-3_May12.doc • The Net Centric Checklist is excerpted from the compliance appendix of the NCOW RM • Modular Open System Approach (MOSA) Progarm Assessment and Rating Tool (PART) • Open Systems Joint Task Force (OSJTF) • MOSA PART is a questionnaire designed to provide a consistent approach to rating programs tailored for DoD.The MOSA PART is an analytic tool that evaluates responses to a set of interrelated questions to provide acquisition program executives with an objective and evidence-based assessment of the degree that MOSA is implemented in a program.

4. NCAT Strategy • Develop a prototype tool to help our members understand and comply with the DoD’s Net Centric approach. • Use what we learn to provide feedback and industry experience to NII as they evolve the NCOW RM and its compliance tools • Utilize existing authoritative works to enable rapid, flexible development using widely used and easily available tools and execution environment • NCOW RM • Net Centric Checklist • MOSA PART • Excel • Our initial effort populated the MOSA PART questionnaire with the Net Centric Checklist

5. Lessons Learned in First Spiral • Original PART UI was somewhat confusing • Did not cleanly separate planned from achieved • The scale of answers was fixed and not specific to each question • Questions are complex and need easily available explanations • Not applicable (N/A) was not cleanly factored from the answer choices • Organization of questions was not consistent with the Net Centric Checklist • Second Spiral addressed these issues • Created new UI buttons to select from pre formatted answers that are specific to each question – multiple choice and N/A • Two ratings to each question – Planned and Achieved • Included original checklist questions for detailed explanations

6. NCAT Parts

7. Illustration of the Assessment – Basic Bar Graph

8. Proposed Features and Solutions • Include multiple evaluations/presentation formats based on questionnaire • Enable user to customize weighting of questions to fit different types of systems • Extend questionnaire to include programmatic evaluation • Similar to other instantiations of checklist questionnaires • Two features are being incorporated into the current edition • Enable historic tracking and progress graphs so that tool can be used as a progress measure as well as snapshot • Develop response directed questioning • If ‘yes’ then ask one sub-set of questions - if ‘no’ as a different sub-set • Incorporate branching into evaluation • Refer to NCAT for demo for present form>>>>>>>

9. Issues to be Resolved in this Spiral • Need “timely” and “strategic” comments from membership to improve the tool • The tool is in spreadsheet and will migrate to a Web-based tool • Do we need to add new categories ( beyond those in the NCOW RM) to the checklist? • How well does this checklist really address NCOW RM (which itself is FOUO)? • We need to organize efforts to test and validate the questionnaire against ongoing programs

10. Questionnaire Issues • Multiple reports are required for multiple viewpoints • Compliance to NCOW • Trade-offs with Quality Attributes • Presentation formats for domain specific questions should be programmable to help evaluator design meaningful presentation (spider charts, bar chart, line chart, etc.) scores to compare against – Needs to planned into future releases Bottom Line – This is your tool – feedback is necessary for success

11. Collaboration Plans • Monitor government feedback on their use of the NetCentric Checklist, CISA COCOM NetCentric Assessment, NAVY FORCENet Checklist, Air Force SISSU effort, etc. as part of JROC and DAB deliberations • Interact with the NCOW RM - DoD/NII in DoD to incorporate their updates and improvements • Provide NCAT to OSJTF for enhancements to MOSA PART effort ( MITRE tasked by OSJTF to update MOSA PART for SoS & NetCentricity features)

12. Issues that need attention by ASA FT and Technical Council as we move forward on NCAT development • NCAT, at this juncture is based on previous release (non-FOUO) of Net-Centric Checklist • NCOW RM is the basis of the Net-Centric Checklist • OSD NII continues to assert that the NCOW RM will remain an FOUO document • There are other documents that should be consulted for their relevance to the NCAT effort • These documents support various service specific checklists, tools, and metrics being developed for measuring Net-Centric compliance • These documents also remain FOUO • KIPs that support the Net Ready KPP specified by CJCSI 6212.01c • SISSU (Secure, Interoperable, Supportable, Sustainable, and Usable) Checklist (AF) – one component of which is the Net-Centric Checklist • Net-Centric Checklist, Ver 2.2 (NII) - Apparently, Versions 2.2 and 3.0 have been set-aside by the DOD CIO. Version 2.1.3 is the baseline for DoD. • Note – FOUO seems to be designated for “drafts” of documentation (not approved for official guidance yet) or versions that must have limited distribution due to the sensitive nature of the documentation

13. Resource Status • 5 NCAT WG members actively contributing • Weekly ( or at least bi-weekly) teleconferences • Participation in e-mail discussions • Encourage additional members to help with “domain” knowledge of questions and applicability • International viewpoints need to brought into the NCAT – we encourage participation • Other Working groups need to review and determine how their interests can be reflected in NCAT (e.g., NIF, Tri-Services, Services and Information Interoperability, etc.)