1 / 54

NDIA Systems Engineering Division Annual Planning Meeting 2013 Summary and 2014 Plans Action Plan Status Committee Repor

NDIA Systems Engineering Division Annual Planning Meeting 2013 Summary and 2014 Plans Action Plan Status Committee Reports. December 11-12, 2013. With placeholders for inputs from OSD and Services - TBD. Introduction. Status of 2013 NDIA SE Division activities Relative to 2013 Plan

jon
Download Presentation

NDIA Systems Engineering Division Annual Planning Meeting 2013 Summary and 2014 Plans Action Plan Status Committee Repor

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NDIA Systems Engineering DivisionAnnual Planning Meeting2013 Summary and 2014 PlansAction Plan StatusCommittee Reports December 11-12, 2013 With placeholders for inputs from OSD and Services - TBD

  2. Introduction • Status of 2013 NDIA SE Division activities • Relative to 2013 Plan • SED plan from 12/12 meeting, briefed at 2/13 meeting • Committee Reports • 2013 Accomplishments • 2014 Plans

  3. NDIA SED 2013 Tasks- Derived from DoD/Services Initiatives (Voice of the Customer) • ASA (ALT) • OCSE • Dev Planning • SE Workforce • SE methods/ tools • Reliability • Security protection • ExportabiIity • Soldier/Small Unit Challenge • DASD (SE) • ERS • Anti-Counterfeit • Risk Mgmt • Schedule risk • Supplier risks • Security design • SE models (MBx) • SoSE Verbal Inputs Provided • SAF AQRE • Policy (levels) • Process • Tradespace • Architecture • Cost Assessment • Specialty Engrg • People / Workforce (specialists vs. generalists) • DASN (RDT&E) • SEP • Agile in acquisition • SoS mission reqts • R&D assessment

  4. NDIA SED 2013 Action Plan Status

  5. NDIA SED 2013 Action Plan Status

  6. NDIA SED 2013 Action Plan Status

  7. NDIA SED 2014 Actions

  8. NDIA SED 2014 Actions

  9. NDIA SED 2014 Actions

  10. Committee Inputs- 2013 Task Progress / 2014 Task Plans • Architecture (Arch) • Automatic Test (ATC) • Development Test and Evaluation (DT&E) • Education and Training (E&T) • Enterprise Health Mgmt (EHM) • Environment, Safety, and Occupational Health (ESOH) • Human-System Integration (HSI) – no slides • Joint Committee on Systems Engrg in Manufacturing (JCSEM) – no slides • Life Cycle Support (LCS) – no slides • Modeling and Simulation (M&S) • Mission Analysis (MA) • Systems Engineering Effectiveness Committee (SEEC) • Software Committee • System of Systems (SoS) • Systems Security Engineering (SSE)

  11. Architecture Committee – 2013 Task Summary • Meetings: • 7 Virtual meetings • 1 Face-to-face meeting • SE Conference: • 15 papers • Accomplishments: • Completed the DoDAF Recommendations Report • Delivered it to Steve Henry • Posted on the website • Presented Architecture Metrics results (worked in 2012) at the INCOSE Symposium • Participated in SEI SoS Architecture workshop • Received architecture informational briefings / demos from C.Gedo, Architecture Stnds Group (DISA & GTRI), and on J6 Architecture portal from B.Piazza (JS J6) • In-work: • Reviewing whitepaper “Role of Architecture to Influence the Development Planning Tradespace” with the Mission Analysis subcommittee • Open system architecture discussions with Dr. Flietz (AFRL)

  12. Architecture Committee - 2014 Task Plan • Proposed 2014 Tasks: • Complete joint Architecture/Mission Analysis white paper “Role of Architecture to Influence the Development Planning Tradespace” • Investigate status and make proposals around subject of Open System Architectures • Initiate joint discussions with Test Committee • Deliverables/Products • “Role of Architecture to Influence the Development Planning Tradespace” whitepaper • Schedule / Resources • Release whitepaper 1Q • Initiate Open System Architecture discussions 1Q • Initiate joint Architecture – Test discussions 1Q • Issues / Concerns: • Availability of committee members

  13. Automatic Test Committee – 2013 Status

  14. Automatic Test Committee - 2014 Task PlanProjects Working Group • Proposed 2014 Tasks: • Government/Industry Liaison Reports • Detail, Staff, Schedule & Initiate new ATC tasks: • Industry Survey for Smart Automatic Test System Opportunities • Software Architecture for Synthetic Instrumentation • Deliverables/Products • Semi-annual Reports posted on ATC Web-site: • US Army ATS Liaison Report • USAF ATS Liaison Report • US Navy/USMC-Air ATS Liaison Report • USMC-Ground ATS Liaison Report • DoD AMB Liaison Report • Commercial ATS Liaison Report • Deliverables from new tasks: TBD • Schedule / Resources • Status/Review/Present at regular ATC meetings: • March/April (adjacent DoD ATS Management Board (AMB) Meeting – Gov’t participating committee) at NDIA HQ • September in St Louis (AUTOTESTCON) • December with SED (TBD) • No resource issues or support anticipated • Issues / Concerns: • Only one concern specific to ATC: • Reduced defense budgets and fallout from the Las Vegas GSA meeting has had a significant impact on Government (AMB) participation in many events involving travel – not just NDIA - including holding virtual AMB meetings. • Response: • ATC is holding its meeting at NDIA HQ adjacent AMB scheduled meetings, and will remain flexible in scheduling our meetings to maximize attendance, participation, etc.

  15. Developmental Test and EvaluationCommitteeStatus and Plans December 2013 Beth Wilson, Raytheon Steve Scukanec, Northrop Grumman Industry Co-Chairs

  16. DT&E Committee2013 Status

  17. DT&E Committee - 2014 Task PlanProjects Working Group • Proposed 2014 Tasks: • Cyber Testing Project • Chief Developmental Tester Project • Collaboration with IEEE Reliability • Joint meetings with SED Committees • Modeling and Simulation • Architecture • Systems Security Engineering • Education and Training • Deliverables/Products • Cyber Testing Recommendations white paper • Chief Developmental Tester white paper • Candidate KSABs for role (E&T) • Industry interaction recommendation • Tutorial on role for SE conference • Inputs to reliability standards effort • Joint meeting notes • Schedule / Resources • Cyber Testing Draft Recommendations Feb 2014 • Weekly teleconferences started 12/3/2013 • Full white paper later in 2014 • Chief Developmental Tester • Joint meeting with E&T with SED meeting • Tutorial at SE conference • Reliability standards input for Jan/Aug meetings • Test as a Stakeholder joint meetings with SED • Modeling and Simulation • Architecture • Systems Security Engineering • Issues / Concerns: • Industry engagement • NDIA T&E conference may be cancelled (Seattle in March) • ITEA conference (Denver in September) may be postponed and moved

  18. Summary of DT&E Committee2013 Results/2014 Plans

  19. DT&E Committee 2012-2013Statistical Test Optimization • 2012 SE Conference summit/workshop thread • Tutorials on Monday 10/22 • Presentations on Wednesday 10/24 • Synthesis Panel on Wednesday 10/24 • Follow-on Results: • DT&E Committee white paper • ITEA Journal article • SE Conference track Oct 2013 2012 - 2013

  20. DT&E Committee 2012-2013Leading Indicator Metrics • 2012 SE Conference workshop • Follow-on workshop with System Performance Measurement WG • Focused on Requirements Verification not addressed in first report • Potential Measures Identified: • System Maturity Level • Verification Requirement Maturity • Technical Measures and Stakeholder Need 2012 - 2013

  21. DT&E Committee 2012-2013Reliability Testing • Reliability Testing Focus • Software reliability standard • Cyber testing initiatives • Data for reliability growth measures • Input/Feedback Opportunities: • IEEE meetings January and August • Periodic revision periods Project Leads: Lou Gullo, TazDaughtrey

  22. DT&E Committee 2013-2014Cyber Testing Project Lead: Dave Desjardins Joint Project with ITEA

  23. DT&E Committee 2013-2014Chief Developmental Tester The Secretary of Defense shall require that each major defense acquisition program be supported by— ‘‘(A) a chief developmental tester; and ‘‘(B) a governmental test agency, serving as lead developmental test and evaluation organization for the program. • Coordinating DT&E activities • Insight into Contractor activities • Oversee T&E activities • Inform government PM about contractor DT&E results Project Lead: Joe Manas

  24. Backup DT&E Committee Results 2008-2013

  25. Summary of DT&E Committee Test and Evaluation Efforts

  26. Summary of DT&E Committee Systems Engineering Efforts

  27. DT&E Committee Current Structure Since 2010 NDIA Systems Engineering Division NDIA Test and Evaluation Division SED Committee Collaborative Projects T&E Themes For Conferences NDIA Industrial Committee on Test and Evaluation (ICOTE) NDIA DT&E Committee Project Results DT&E Themes SE Themes & Issues T&E Themes & Issues NDIA Systems Engineering Conference (Q4) Collaborative Projects NDIA Test and Evaluation Annual Conference (Q1) Periodic T&E Committee-Focused Conferences Cooperation with other NDIA Divisions DT&E Committee Focus: T&E initiatives aligned with SE, DT&E

  28. DoD T&E Policy Study • August 2006: DT&E Committee Kickoff • Policy Study: • “Improving T&E in the DoD Acquisition Process” • Industry T&E policy recommendations • Workshops: • August 2007 • January 2008 • Focus Areas: • 1. Earlier contractor and tester involvement • 2. Integrated DT/OT and DT operational relevance • 3. Suitability • April 2008: Report Summarized Results: • 10 Findings • 15 Recommendations 2006-2008

  29. Integrated Testing (CT/DT/OT) Implementation Framework 2008-2010

  30. Software Test and EvaluationSoftware Summit 2009

  31. RFP Language Industry Comments for Update: Incorporating Test and Evaluation Into Department of Defense Acquisition Contracts Recommendations from SW Summit 2010-2011

  32. Test and Evaluation forSystems of Systems 2009: “Sleepless Nights” List of Issues 2010: “Sominex” Resulting Initiatives 2010: Workshop 2011: Best Practices Wave Model 2012: Final Report 2009-2012

  33. Effective Use of Modeling and Simulation for Test and Evaluation • Joint Meeting in August 2011 • Distributed Testing, the Joint Mission Environment Test Capability (JMETC) and the Test and Training Enabling Architecture (TENA) • DoD M&S Community of Interest Data Management Working Group • LVC Architecture Roadmap Implementation (LVCAR‐I) Gateways Effort Applicability to T&E • OSD T&E Working Group • Raytheon Presentation on M&S for T&E • Potential Topics for November AMSWG Meeting 2011

  34. Modeling & Simulation forDistributed Testing • Benefits: • Find integration issues earlier • Test to learn in ‘safe’ environment • Protect proprietary information • Facilitate DT to OT transition • Increase performance testing range in operating environments • Support end to end studies throughout the program • Barriers: • Security • Lack of persistent network • Early consideration of technical issues • Perceived value • Disconnect between the communities (M&S and T&E) • Recommendations: • Harmonize the standards for M&S and Test for the life cycle perspective (HLA, TENA, Metadata) • Create a framework for reusing and repurposing M&S through the product model • Establish M&S as part of statistical test design • Determine what tests are conducted to acquire data for model validation. • Fewer test events with better models. • Recommend the use of M&S to do I&T • Recommend establishment of JMETC as a persistent node for industry to engage in MBDI&T Joint Meeting August 2012 Joint Track SE Conference October 2012 2012

  35. Education and Training Committee – 2013 Task Summary • Established a Joint Competency Working Group (JCWG) • Includes members from E&T Committee and INCOSE Competency WG • Main objective is to evolve to a globally accepted* and marketed standard competency framework, based on systems engineering effectiveness, that can be used to produce competency models tailored to the needs of the customer organizations. • Developed white paper and proposal to evolve current INCOSE UK Competencies Framework along four paths: • Evolve to an SE role-based competency framework that is extensible, scalable, and tailorable by the customer organization. • Evolve by adding the concept of classes. This is where the competencies achieve their interdependence with each other. • Evolve by including a Professional Class that covers leadership and soft skill competencies. • Evolve by adding a new level of proficiency called the Senior Practitioner to the existing four levels in the INCOSE UK Competencies Framework. * Approved for release as an INCOSE product.

  36. Education and Training Committee - 2014 Task Plan • Proposed 2014 Tasks: • Develop crisp definitions of Knowledge, Skills, Abilities, Behaviors and Experience. • Develop better definitions of the six classes. • Refine definition of the Senior Practitioner level. • Develop better definitions for each version (0.5, 0.75, and 1.0). • Explore how system complexity affects SE Roles. • Develop use cases. • Ensure consistency with INCOSE Certification program and SE Handbook. • Deliverables/Products • Competencies Framework V 0.5 • Competencies Framework V 0.75 • Competencies Framework V 1.0 • Conference Papers and Presentations • Final Report • Issues / Concerns: • Few outside UK use framework – Need to seek broader buy-in early and throughout development process and show how we can use evolved competencies framework to clarify what a SE is, what they do, and why they are important. • Project may face analysis paralysis – Need to set reasonable POA&M, stick with schedule, discuss, move forward, exhibit strong leadership along the way. • Schedule / Resources: • Two weeks before INCOSE IW 2014, send draft Version 0.5 to JCWG members. • Conduct working session at IW 2014 to finalize Version 0.5. • Conduct working session at IS 2014 to finalize Version 0.75 • Ultimate Goal: Produce Version 1.0 by IW 2015.

  37. EHM Committee – 2013 Task Summary

  38. (EHM Committee - 2014 Task Plan • Proposed 2014 Tasks: • working with OSD and rewrite of Integrated Diagnostic handbook • Deliverables/Products • Review/Comments of Integrated Diagnostics Handbook as it progresses • Schedule / Resources • Status/Review/Present at regular EHM meetings: • March/April EHM Meeting • September EHM Meeting • December with SED • No resource issues or support anticipated • Issues / Concerns: • No Risks, Barriers or Concerns specific to EHM; However, reduced defense budgets has had a significant impact on Government, and defense contractor participation in many events

  39. ESOH Committee – 2013 Task Summary • Completed 2013 Tasks: • Supplemented the new Defense Acquisition Guidebook Chapter 4, Systems Engineering • Teaming with DoD Acquisition ESOHIPT • “Handbook for Human Systems Integration (HSI) & Environment, Safety, and Occupational Health (ESOH) in Pre- Milestone A Analysis of Alternatives (AoA) and Joint Capabilities Integration and Development System (JCIDS) Activities,” published October 2013 • “Guide to ESOH in the Systems Engineering Plan (SEP); Programmatic Environment, Safety, and Occupational Health Evaluation (PESHE); and National Environmental Policy Act (NEPA)/Executive Order (EO) 12114 Compliance Schedule,” published October 2013 • Coordinated with Aerospace Industrial Association (AIA) on rewrite of National Aerospace Standard 411 (NAS411) • Teaming with DoD Acquisition ESOH IPT • Goal of rewrite: provide detailed implementation instructions to contractors on implementation of the MIL-STD-882E, DoD Standard Practice for System Safety, Task 108, "Hazardous Materials Management Plan" • NAS 411, “Hazardous Materials Management Program,” published 30 September 2013 • NAS 411-1, “Hazardous Materials Target List,” published 30 September 2013 • Key element: List of hazardous materials categorized per Task 108 as Prohibited or Restricted • Planned the NDIA SE Conference ESOH Track • Combined HSI and ESOH Track and coordinated abstracts with the HSI Committee • Six abstracts to address MRAP Lessons Learned for Rapid and Urgent Program ESOH management, streamlining ESOH documentation initiatives, and updates on DoD Policy

  40. ESOH Committee - 2014 Task Plan • Proposed 2014 Tasks: • Focus on improving implementation of current DoD Acquisition SE ESOH policy • Continue work with AIA on NAS 411-1 list of Tracked materials • Work with DSOC/SOH Council on memo regarding 15 Nov 2013 Army CAE memo on ESOH risk management and reporting • MIL-STD-882E implementation work • Develop mechanism and collect Industry and Government feedback on policy and guidance • Quarterly webinars • Deliverables / Products • List of Tracked Hazardous Materials • Memo co-signed by P&R and DUSD (I&E) • MIL-STD-882Ecorrections and DIDs • Comments on improving DoD ESOH policy and guidance, and MIL-STD-882E • Webinars • Issues / Concerns: • Agreement between DoD and AIA on Tracked list • Time for staffing and advocacy for proposed P&R and DUSD (I&E) memo • Funding for MIL-STD-882E update development • Time and funding for questionnaire development • OSD support • Schedule / Resources • NAS 411-1 list of Tracked materials – ECD Dec 2014/funding by AIA and DoD participants • Memo regarding 15 Nov 2013 Army CAE memo – ECD Jun 2014/funding by DoD participants • MIL-STD-882Eimplementation work – TBD • Develop mechanism and collect Industry and Government feedback on policy and guidance – ECD Dec 2014/funding by participants • Quarterly webinars – ECD June 2014/DoD funded

  41. Modeling and Simulation Committee – 2013 Task Summary • Final report on M&S use assessment • Briefing on product given at October 2013 Systems Engineering Conference • Key spreadsheet completed and shared with ODASD/SE/SA liaison • Final report draft in progress – to be completed in mid-December • Committee meetings • Meetings held in February, April, June, and August 2013 • Presentations posted on committee web site • Report on essential elements of a System Model • Subcommittee formed in August 2013 • Expect final report in July 2014

  42. Modeling and Simulation Committee - 2014 Task Plan • Proposed 2014 Tasks: • Complete study to determine essential elements of a “System Model” that evolves over the life cycle (started in 2013) • Provide continuing situational awareness for M&S Committee members (ongoing) • Motivate submittal of abstracts to populate full-length M&S track at 2014 SE Conference • Deliverables/Products • Final report on essential elements of a System Model • Committee meeting presentations posted on committee web site • Schedule / Resources • Final report on essential elements of a System Model – July 2014 • Have some volunteers; need to recruit more • Committee meetings in February, April, June, and August 2014 • Need organizations to volunteer to host • Issues / Concerns: • Finding volunteers to work on unfunded study efforts is getting much more difficult in current fiscal environment. • Continuing Government restrictions on travel and fees for professional society activities is creating difficulties in maintaining perception of value of society activities to industry participants.

  43. Mission Analysis Committee – 2013 Task Summary • Collaborated with ASD R&E/Research on the integration of S&T/IR&D • Industry involvement in the Multidisciplinary University Research Initiative (MURI) process • Increase Industry awareness and use of Defense Innovation Marketplace • Developed draft whitepaper on “The Role of Architecture in the Development Planning Trade Space” with the Architecture Committee • Collaborated with the Government DPWG on Development Planning Activities Models • Best Practices, Techniques, Methodologies, and Tools • Completed all actions from June 2012 NDIA DPWG Workshop, except: • Intellectual Property as a barrier to successful DP • OCI as a barrier to successful DP • Final DPWG Workshop Report not submitted

  44. Mission Analysis Committee - 2014 Task Plan • Proposed 2014 Tasks: • Support the Government DPWG and ASD R&E/Research on an as needed basis • Organizational Conflict of Interest (OCI) Task • Collaborate with the Government DPWG to address OCI • Develop a clear and concise understanding of the FAR / DFARS language • Communicate findings across Gov’t/ Industry • Complete the “Role of Architecture in Development Planning” Whitepaper with the Architecture Committee • Deliverables/Products • “Role of Architecture in Development Planning” Whitepaper w/ the Architecture Committee • Final Report on OCI Task findings and recommendations • Schedule / Resources • “Role of Architecture in Development Planning” Whitepaper – April 2014 • OCI Findings and Recommendations – June, 2014 • OCI Final Report – August, 2014 • Issues / Concerns: • Diminishing number of contributing committee members • Developing Planning instantiation across Government and Industry has been largely successful. It is questionable as to the need to continue the Mission Analysis Committee. Resources may be better utilized elsewhere.

  45. SE Effectiveness Committee– 2013 Task Summary • Widely disseminated the results of the SE Effectiveness Study through numerous channels • Released detailed survey results report • Discussed follow-on work ideas with DASD(SE) • SE self-assessment tool for programs • “Lightweight” Pre-PSR program assessment

  46. Systems Engineering Effectiveness Committee – 2014 Task Plan • Proposed 2014 Tasks: • Task 1: Develop a means of measuring risk management process value • - Enable the evaluation of risk management in programs • - Enable the comparison of alternate risk management paradigms • - Enable the improvement of risk management processes • Task 2: “Get this* out to the SAEs.”, (awaiting further instructions from DASD (SE) – some possibilities listed below) • - Periodic self-assessment of programs • Pre- Program Support Reviews (PSR) assessment of programs • Task 3: Establish collaboration effort with INCOSE SE Effectiveness Working Group • * The SE Effectiveness Study results (per The Honorary Frank Kendall, USD (AT&L)) • Deliverables/Products • Task 1: Presentation at Annual SE Conference • Task 2: Report out at year-end Planning Meeting • Task 3: TBD • Schedule / Resources • Task 1: Oct2014 • Task 2: Dec 2014 • Task 3: TBD • Task 1 & 2: Assigned liaison from DASD (SE) office • Issues / Concerns: • Working agreement with INCOSE SE Effectiveness Working Group

  47. SoS SE Committee 2013 Task Summary • Committee inputs to TTCP Product addressing SoS at key points in the systems development process • February 2013: Initial presentation of TTCP draft • March 2013: Inputs from committee members on TTCP draft • November 2013: Shared Updated Product, with request for added inputs • December 2013: Inputs coming in now • SoS Webinar Series: SoS Engineering Collaborators Information Exchange (SoSECIE) • 14 Webinars, each with between 40-100 participants • Presentations are posted on SoS SE Website http://www.acq.osd.mil/se/outreach/sosecollab.html • Committee participation SEI workshop on SoS SW Reuse Patterns • Committee Meetings focused on Issues of Interest • February - TTCP Product Review • April – T-Area-SoS – EU SoS Research Agenda Development Projects • July – Army Victory Architecture (with Architecture Committee) • August – M&S and SoS (with M&S Committee) • Six SoS tracks at SE Conference, most joint with other committees • Drafted 15288 annex for systems of systems accepted for revision

  48. SoS Committee2014 Task Plan • Deliverables/Products • Feedback on inputs to TTCP work product • Presentations on SoS SE Committee Area on NDIA Website • Webinars slides posted on DASD SE site • Proposed 2014 Tasks • Input and Feedback from TTCP • Regular committee exchanges • Focus on new and emerging Service SoS Initiatives • Joint meeting with SSE on SoS/SSE • SoS Webinar Series: SoS Engineering Collaborators Information Exchange (SoSECIE) • Schedule / Resources • Bi-Monthly committee exchanges • Bi-Weekly Webinars • Issues / Concerns • None

  49. Systems Security EngineeringCommitteeStatus and Plans December 2013 Holly Coulter Dunlap, Raytheon Beth Wilson, Raytheon Industry Co-Chairs

  50. SSE Committee2013 Status: NEW

More Related