1 / 25

P1516.4: VV&A Overlay to the FEDEP 20 September 2007 Briefing for the VV&A Summit

P1516.4: VV&A Overlay to the FEDEP 20 September 2007 Briefing for the VV&A Summit. Simone Youngblood M&S CO VV&A Proponency Leader 240-228-7958 Simone.Youngblood@jhuapl.edu. Current VV&A Standards Initiatives. VV&A FEDEP Overlay VV&A Templates. VV&A FEDEP Overlay. Background.

troy-jenkins
Download Presentation

P1516.4: VV&A Overlay to the FEDEP 20 September 2007 Briefing for the VV&A Summit

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. P1516.4: VV&A Overlay to the FEDEP20 September 2007Briefing for the VV&A Summit Simone YoungbloodM&S CO VV&A Proponency Leader 240-228-7958Simone.Youngblood@jhuapl.edu

  2. Current VV&A Standards Initiatives • VV&A FEDEP Overlay • VV&A Templates

  3. VV&A FEDEP Overlay

  4. Background • VV&A Product Development Group (PDG) initiated in Spring 2004 • VV&A Overlay Drafting Group had first meeting in Winter 2004 • Draft overlay standard development progress briefed in Spring 2004, Fall 2004, Spring 2005, Summer 2005 and Fall 2005 • Draft overlay standard released to the VV&A PDG for comment on 31 March - 15 May 2006 • Drafting Group met to develop comment resolution recommendations on 5-9 June 2006 • VV&A PDG met to approve comment resolutions on 28 June 2006 • Revised draft standard released to the VV&A PDG for comment on 4 August - 4 September 2006 • Drafting Group met to develop comment resolution recommendations on 5-9 September 2006 • VV&A PDG meets to decide recommended comment resolutions on 14 September 2006

  5. Scope • Identifies and describes the recommended verification, validation and accreditation (VV&A) processes that one should perform to assure the acceptability and utility of federations for particular purposes. • Identifies and describes the information feeding and resulting from those processes as well as the relationships between the High Level Architecture (HLA) Federation Development and Execution Process (FEDEP) and the VV&A processes and their respective information products . • Defines those terms uniquely needed to characterize the FEDEP VV&A overlay (e.g., federation conceptual model, interoperability, representational requirements). • Takes special care to use and build upon existing standards, standard terms and their definitions whenever possible. • Focuses upon the VV&A processes that apply to federations. • Does not address the VV&A processes associated with individual simulations (federates) but does consider using the information produced by those processes. • Does not describe individual techniques that one might employ to execute the VV&A processes for federations.

  6. Draft VV&A Overlay Standard Outline • Overlay: Top Level View • Overlay: Detailed View • Phase 1 - Verify Federation Objectives • Phase 2 - V&V Federation Conceptual Model • Phase 3 - Verify Federation Design • Phase 4 - Verify Federation Development Products • Phase 5 - Validate & Accept Federation • Phase 6 - V&V Federation Outputs • Phase 7 - Consolidate Federation VV&A Products • Conclusion • Introduction • Scope • Purpose • References • Definitions, Abbreviations & Acronyms • Definitions • Abbreviations & Acronyms • Roles & Responsibilities • Introduction to VV&A Roles & Responsibilities • Role Mapping to Overlay Activities • Assumptions & Tailoring • Assumptions • Tailoring

  7. Some Terms Important to the VV&A Overlay The drafting committee encountered several terms while drafting the VV&A Overlay to the FEDEP that were relevant to federation VV&A: • interoperability • parameter • referent • representational requirements • results sampling strategy * • risk • scenario • substantive interoperability • uncertainty * • traceability • validation • verification • acceptability criteria * • acceptance * • accreditation • credibility * • error characteristics * • federate • federation • federation conceptual model • federation objectives • federation object model • federation requirements * • fidelity * The drafting committee took special care to use the definitions of terms in the FEDEP and the well respected definitions from other sources whenever possible. They only judiciously crafted new definitions when no existing definition captured the meaning of the terms as they were used in the overlay (marked by an *).

  8. Overview of the Draft Overlay Activities • Phase 4 - Verify Federation Development Products • Activity 4.1 - Support Developing FOM • Activity 4.2 - Contribute to Verifying FOM • Activity 4.3 - Support Establishing Federation Agreements • Activity 4.4 - Contribute to Verifying Federation Agreements • Activity 4.5 - Support Implementing Federate Designs • Activity 4.6 - Support Implementing Federation Infrastructure • Activity 4.7 - Contribute to Verifying Federation Infrastructure • Activity 4.8 - V&V Federation Data Sets • Phase 5 - Validate & Accept Federation • Activity 5.1 - Support Planning Federation Execution • Activity 5.2 - Support Integrating Federation • Activity 5.3 - Contribute to Verifying Integrated Federation • Activity 5.4 - Support Testing Federation • Activity 5.5 - Validate Integrated Federation Results • Activity 5.6 - Perform Acceptance Assessment Phase 6 - V&V Federation Outputs • Activity 6.1 - Support Executing Federation • Activity 6.2 - Contribute to Verifying Raw Execution & Derived Output • Activity 6.3 - Validate Federation Outputs Phase 7 - Consolidate Federation VV&A Products • Activity 7.1 - Support Analyzing Data • Activity 7.2 - Prepare Federation VV&A Products for Reuse • Phase 1 - Verify Federation Objectives • Activity 1.1 - Support Identifying User/Sponsor Needs • Activity 1.2 - Plan Accreditation Activities • Activity 1.3 - Support Developing Federation Objectives • Activity 1.4 - Contribute to Verifying Federation Objectives • Activity 1.5 - Assemble Federation Referent • Activity 1.6 - Define Federation Acceptability Criteria • Activity 1.7 - Plan V&V Activities • Phase 2 - V&V Conceptual Model • Activity 2.1 - Support Developing Federation Scenarios • Activity 2.2 - Contribute to Verifying Federation Scenarios • Activity 2.3 - Support Developing Federation Conceptual Model • Activity 2.4 - Contribute to Verifying Federation Conceptual Model • Activity 2.5 - Validate Federation Conceptual Model • Activity 2.6 - Support Developing Federation Requirements • Activity 2.7 - Contribute to Verifying Federation Requirements • Phase 3 - Verify Federation Design • Activity 3.1 - Support Selecting Federates • Activity 3.2 - Support Preparing Federation Design • Activity 3.3 - Contribute to Verifying Federation Design • Activity 3.4 - Support Preparing Federation Development and Execution Plan

  9. Define Federation Objectives Perform Conceptual Analysis Design Federation Develop Federation Analyze Data & Evaluate Results FEDEP Plan, Integrate & Test Federation Execute Federation & Prepare Outputs Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 FEDEP INFORMATION & PRODUCTS Verify Federation Development Products • Verify Federation Objectives V&V Federation Conceptual Model Verify Federation Design Validate & Accept Federation V&V Federation Output Consolidate Federation VV&A Products Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 6 Phase 7 • Verify the Federation Objectives with the User /Sponsor. • Define the Federation Acceptability Criteria. • Assemble the Federation Referent. • Plan the VV&A activities. • Verify the Federation Conceptual Model internally. • Validate the Federation Conceptual Model against the Federation Referent and against the Federation Acceptability Criteria. • Verify the Federation Design internally. • Verify the Federation Design against the Federation Conceptual Model. • Verify the federation development products internally. • Verify the federation development products against the Federation Conceptual Model. • Validate the integrated federation results against the Federation Acceptability Criteria and against the Federation Referent. • Formulate federation acceptance/ accreditation recommen-dations. • Verify the federation outputs internally. • Validate federation output against the Federation Acceptability Criteria and against the Federation Referent. • Collect all VV&A products and lessons learned. • Assemble them into a VV&A package to support federation reuse. VV&A Overlay to the FEDEP Top Level View of the VV&A Overlay to the FEDEP

  10. Verify Federation Objectives 1.0 Federation Acceptability Criteria, Federation Referent, VV&A Plans V&V Federation Conceptual Model 2.0 federation conceptual model V&V results Verify Federation Design 3.0 federation design verification results Define Federation Objectives Verify Federation Development Products 4.0 federation development verification results verified Federation Objectives Perform Conceptual Analysis Validate & Accept Federation 5.0 federation validation and acceptance results validated Conceptual Model Design Federation V&V Federation Output 6.0 verified Federation design Develop Federation Consolidate Federation VV&A Products 7.0 verified Federation development products federation output V&V results Plan, Integrate and Test Federation accepted Federation Execute Federation and Prepare Outputs Federation VV&A products validated Federation output Analyze Data and Evaluate Results Detailed View of the VV&A Overlay to the FEDEP

  11. Verify Federation Development Products Verify Federation Objectives Validate & Accept Federation V&V Federation Output Consolidate Federation VV&A Products V&V Federation Conceptual Model Verify Federation Design 1.0 2.0 3.0 4.0 5.0 6.0 7.0 Support Identifying User/Sponsor Needs Support Developing Federation Scenarios Support Selecting Federates Support Developing FOM Support Planning Federation Execution Support Executing Federation Support Analyzing Data Contribute to Verifying FOM Contribute to Verifying Raw Execution and Derived Output Prepare Federation VV&A Products for Reuse Support Preparing Federation Design Contribute to Verifying Federation Scenarios Support Integrating Federation Plan Accreditation Activities Support Establishing Federation Agreements Support Developing Federation Conceptual Model Contribute to Verifying Integrated Federation Contribute to Verifying Federation Design Support Developing Federation Objectives Validate Federation Outputs Contribute to Verifying Federation Agreements Support Testing Federation Contribute to Verifying Federation Conceptual Model Support Preparing Federation Development & Execution Plan Contribute to Verifying Federation Objectives Support Implementing Federate Designs Validate Integrated Federation Results Validate Federation Conceptual Model Support Implementing Federation Infrastructure Assemble Federation Referent Perform Acceptance Assessment Support Developing Federation Requirements Contribute to Verifying Federation Infrastructure Define Federation Acceptability Criteria Contribute to Verifying Federation Requirements V&V Federation Data Sets Plan V&V Activities Tasks Associated with Each VV&A Overlay Phase

  12. Plan Accreditation Activities Plan V&V Activities 1.2 Support Identifying User/Sponsor Needs Support Developing Federation Objectives 1.7 Define Federation Acceptability Criteria 1.1 1.3 Federation Accreditation Plan Federation V&V Plan 1.6 Assemble Federation Referent Federation Acceptability Criteria 1.5 Federation Referent Contribute to Verifying Federation Objectives User/Sponsor Needs 1.4 Identify User/Sponsor Needs Develop Objectives verified Federation Objectives Phase 1 - Verify Federation Objectives

  13. Support Developing Federation Scenarios Support Developing Federation Requirements 2.1 2.6 Contribute to Verifying Federation Scenarios verified Federation Scenarios verified Federation Objectives Develop Scenario 2.2 Contribute to Verifying Federation Requirements Support Developing Federation Conceptual Model Develop Federation Requirements 2.7 verified Federation Requirements 2.3 Validate Federation Conceptual Model Contribute to Verifying Federation Conceptual Model 2.5 validated Federation Conceptual Model Develop Federation Conceptual Model 2.4 • Federation Referent • Federation Acceptability Criteria Phase 2 - V&V Federation Conceptual Model

  14. Support Selecting Federates Support Preparing Federation Design 3.1 3.2 Federation Acceptability Criteria Support Preparing Federation Development & Execution Plans 3.4 Contribute to Verifying Federation Design verified Federation design Select Federates Prepare Federation Design 3.3 updated FDEP Prepare Plan (FDEP) validated Federation Conceptual Model • Federation Accreditation Plan • Federation V&V Plan Phase 3 - Verify Federation Design

  15. Support Developing FOM Support Establishing Federation Agreements Support Implementing Federate Designs Support Implementing Federation Infrastructure 4.1 4.3 4.5 4.6 verified Federation Agreements Contribute to Verifying FOM Contribute to Verifying Federation Agreements verified FOM validated federates Develop FOM Establish Federation Agreements Implement Federate Designs 4.2 4.4 Contribute to Verifying Federation Infrastructure verified federation infrastructure 4.7 Implement Federation Infrastructure Verify and Validate Data Sets • verified federation design products • validated Federation Conceptual Model supporting databases 4.8 verified scenarios Data Sets validated data sets existing federate data sets • Federation Referent • Federation Acceptability Criteria Phase 4 - Verify Federation Development Products

  16. Support Testing Federation Support Integrating Federation Support Planning Federation Execution 5.1 5.2 5.4 Perform Acceptance Assessment verified integrated federation Plan Execution Integrate Federation Test Federation validated Federation 5.6 acceptance recommendation Federation validation results Validate Integrated Federation Results Contribute to Verifying Integrated Federation 5.3 5.5 Federation Referent Federation Acceptability Criteria V&V activity results Phase 5 - Validate and Accept Federation

  17. Support Executing Federation 6.1 Contribute to Verifying Raw Execution and Derived Output Validate Federation Outputs Execute Federation Prepare Federation Outputs federation outputs 6.2 6.3 federation output validation results • Federation Referent • Federation Acceptability Criteria Phase 6 - V&V Federation Output

  18. Support Analyzing Data 7.1 Prepare Federation VV&A Products for Reuse Analyze Data Evaluate and Feedback Results federation outputs analyzed data 7.2 reusable VV&A products V&V activity products Phase 7 - Consolidate Federation VV&A Products

  19. P1516.4 Recommended Practice for VV&A of a Federation, an Overlay to the HLA Federation Development and Execution ProcessInitial Ballot Ballot Open Date: 04/17/2007 Ballot Close Date: 05/17/2007 RESPONSE RATE This ballot has met the 75% returned ballot requirement 55 eligible people in this ballot group. 44 affirmative votes 3 negative votes with comments 0 negative votes without comments 5 abstention votes 52 votes received = 95% returned, 10% abstention APPROVAL RATE The 75% affirmation requirement is being met 44 affirmative votes 3 negative votes with comments 47 votes = 94% affirmative

  20. P1516.4 Recommended Practice for VV&A of a Federation, an Overlay to the HLA Federation Development and Execution ProcessRecirculation Ballot Ballot Open Date: 06/26/2007 Ballot Close Date: 07/06/2007 RESPONSE RATE This ballot has met the 75% returned ballot requirement 55 eligible people in this ballot group. 45 affirmative votes 2 negative votes with comments 0 negative votes without comments 5 abstention votes 52 votes received = 95% returned, 10% abstention APPROVAL RATE The 75% affirmation requirement is being met. 45 affirmative votes 2 negative votes with comments 47 votes = 96% affirmative

  21. Summary of Accomplishments • Developed a succinct and consistent model of the federation VV&A process that guided the the overlay’s formulation • Crafted definitions for several important terms (e.g., federation requirements, credibility) where no relevant definitions existed in other sources • Integrated the results from the preliminary NATO VV&A Overlay with the initial draft of the SISO VV&A Overlay to the FEDEP • Clearly distinguished three important classes of tasks that involve the VV&A Team: • Support - FEDEP tasks in which the VV&A Team should participate • Contribute - verification tasks that the VV&A Team should monitor and supplement where necessary • VV&A unique - tasks for which the VV&A Team are completely responsible • Clarified the VV&A responsibilities for Steps 6 & 7 of the FEDEP, the life after federation acceptance • Reached agreement between international representatives (Canada, France, Germany, Sweden, United Kingdom & USA) on the structure and details of the VV&A overlay to the FEDEP • Ballotted P1516.4 through the IEEE process

  22. VV&A Templates

  23. Mission Statement Enable expanded M&S reuse by building the foundation for consistent V&V information to support accreditation decisions.

  24. Outlines of the Four Core VV&A Documents

  25. Templates Status • Draft MIL Standard entitled • Department of Defense Standard Practice: Modeling and Simulation (M&S) Verification, Validation, and Accreditation (VV&A) Documentation Templates • DSP review closed 17 July 07 • Comment adjudication completed 23 August 07

More Related