A Framework for OT&E Transformation NDIA 20th Annual Test and Evaluation Conference

1. A Framework for OT&E Transformation NDIA 20th Annual Test and Evaluation Conference 4 March 2004 Dr. William G. Lese Vice President Simulation, Analysis & Training Systems Northrop Grumman Information Technology

2. Discussion Topics • Transformation is Underway • OT&E Transformation Complexities • OT&E Transformation Framework • Conclusions

3. Transformation is Underway “We must transform not only the capabilities at our disposal, but also the way we think, the way we train, the way we exercise, and the way we fight.” – Secretary of Defense Rumsfeld • Jointness • Office of Force Transformation Established • Services Transformation Plans • Training Transformation (e.g. JNTC) • Defense Readiness Reporting System (DRRS) • Relating readiness to mission metrics • Joint Experimentation • Knowledge Management OT&E Transformation is Inevitable

4. OT&E Transformation Complexities • Joint OT&E • Concept Definition • Facilities • People • Process (Sequential vs. Iterative) • Test Objectives • Measures and metrics • Data collection • Data Analysis • Cost • Responsibility • OT&E Ability to Address Adaptability and Agility • Moving away from Pass-Fail Mentality

5. OT&E Transformation Complexities • Missing the Mission Focus – Not technology focus • Size of test resources not sufficient for system of systems test • Training Transformation Relationship • Stronger link between joint experimentation and OT&E • MEM (Model-Experiment-Model) • Mission Metrics • Joint Context • Service and Joint OT&E Relationship • Facilities, resources, connectivity • Culture – Knowledge Management “In view of the challenges we face now and in the future, I have decided to change how we develop and execute programs to ensure that our programs serve joint needs and effectively balance current and future risks.” Sec Def. Oct 2003 Memo for Mil Dept Secretaries

6. MNS System Dev and Demo Production and Deployment Support C&T Dev FRP B C A LRIP ITEM DEM-VAL I N D U S T R Y JOINT OPERATIONAL ANALYSIS JT Staff USER RQMT ASSESSMENT Operational Requirements DEM-VAL Academia Industry I N D U S T R Y USER RQMT ASSESSMENT JNTC JOINT OPERATIONAL ANALYSIS Data Archives Data Mart Applications FRAMEWORK FOR TRANSFORMATION-BASED TESTING DOD Acquisition Cycle CONCEPTS EDM FRP ITEM EOA OA (1&2) T E S T IOT&E M & S FOT&E Continuous Testing, Improving, and Learning Spiral TRANSFORMATIONAL CHANGE (DOTMLPF) KM GRID Services Collaborative Environment COCOMs Internet/Intranet Identify & Link SMEs SIPRNet DT & OT Processes Test and Evaluation Enablers Training Vectoring Information Smart Push-Pull Network Agents & Search Engines M & S Experimentation Virtual Collaborative Teams and Communities of Practice

7. Key Features of the Framework • KM Impact on OTE - COPs • Access to experts – databases • Portals/ web based/connectivity • Link to JNTC • Open architecture • Testing is JTF mission based • Link to experimentation • Testing supported by M&S • Testing is spiral and continuous • Feedback to requirements

8. The Framework is a Roadmap for Successfully Transforming OT&E • Joint OT&E • Adaptability and agility • Pass-Fail Mentality • Mission focus • Training Transformation Relationship • Joint Experimentation Link • Service and Joint OT&E Relationship • Culture – Knowledge Management

9. Conclusions • OT&E transformation is inevitable • KM processes and procedures play a critical role in addressing OT&E transformation complexities • Proposed framework is a roadmap for successfully transforming OT&E