Integrated Survivability Assessment (ISA) for OT&E

1. Integrated Survivability Assessment (ISA) for OT&E Cleared for Public Release Distribution Unlimited Ron Ketcham Chairman, JASPO Survivability Assessment Subgroup Naval Air Warfare Center, Weapons Division Code 418100D (760) 939-2363 (DSN 437-2363) FAX: (760) 939-2062 Dave Hall SURVICE Engineering Company 900E N. Heritage Drive, Suite 1 Ridgecrest, CA 93555 (760) 446-4624 X106; FAX: (760) 446-2424; Cell: (760) 382-1618

2. ISA Project • Motivation: comprehensive OT&E system survivability evaluations • Tasking from Joint Aircraft Survivability Program Office (JASPO) • At the request of DOT&E • Objectives: • Combine the results of LFT&E and OT&E into an overall assessment of the survivability of a system under test • Adaptable for use in all phases of system development and fielding from Concept Definition through FOT&E • Make use of appropriate M&S and T&E Resources

3. Elements of Survivability 3. Damage Resistance 2. Crew Protection 4. Signature Reduction 1. Damage Tolerance 5. Defensive ECM/IRCM 13. Tactics & Doctrine Vulnerability (LFT&E) Susceptibility 6. Situational Awareness 12. Policy & ROE Other 11. Mission Planning Outside Factors Affecting Susceptibility 7. Aircraft Performance 10. Off Board Jamming 8. Precision & Standoff Weapons 9. SEAD & DEAD

4. Tasking • Develop a combined survivability OT&E/LFT&E Checklist to evaluate all aspects of the survivability of air vehicles • Develop metrics to evaluate the checklist • Develop an ISA process for measuring those metrics using current JASPO and DOT&E methodologies, T&E range capabilities and tools • Identify deficiencies in the ISA process (M&S, data, T&E, etc.) and develop detailed plans to mitigate them • Exercise the ISA process for one or more acquisition programs

5. Susceptibility Assessment Target Acquisition EW/EC Mission Planning Specification Compliance Sensor Models Signature Prediction Mission- Threat Analysis EW/EC Air-to-Air Combat OT&E/LFT&E SAM Guns Air-to-Ground Missions AAM Lasers Threat Engagement Assessment Mission Survivability Assessment AOA Vulnerability Assessment Requirements Definition Integrated Survivability Assessment Process

6. Integrated Survivability Assessment • A process to evaluate all aspects of system survivability in a coordinated fashion • Using both M&S and T&E resources where appropriate • Including LFT&E and survivability OT&E issues • Perform survivability OT&E and LFT&E in the context of missions and scenarios for the system under test • By using appropriate “vignettes” • Vignette Matrix identifies potential OT&E/LFT&E test conditions • Provides a framework for developing LFT&E and survivability OT&E test plans • Negotiated between PM, Service OT, DOT&E • Provides a roadmap for use of M&S in support of TEMP • Relates to JSF, AIM-9X experience • Developed example matrices of vignettes representing missions & scenarios for three aircraft types • Helicopter, Transport, TACAIR

7. Sample Weights Roles Missions Somalia NEA SWA AFGHAN Urban Forest Desert Mountains Ж X X X 10 Attack Helo CAS Battlefield Interdiction X X 10 Ж 5 X Air Combat Ж Air Mobile Escort X X X 10 Ж CSAR Escort** X X 5 Ж X X X 30 AIR CAV RECCE* Ж Ж X X X 20 SCREEN X X X 10 Target Acq Ж Takeoff & Landing Ж X X X 100 Driving factors Close engagement range, hard to find targets IADS, weather, RF threats, MANPADS, target acq. Flat terrain, clear weather, CB threat High altitude, rough terrain *Most likely mission **Most stressing mission Ж = Most stressing scenario Example Helicopter Vignettes

8. Example Helicopter Vignette • Close Air Support to troop insertion mission in Operation Anaconda • Combined operation with AH-64, UH-60, CH47, CAS assets • Identified LZ location, mission objectives • Rugged terrain • Uncertain threat situation • Uncertainty in threat means that a number of variations in the vignette should be considered • To determine “robustness”, or sensitivity of survivability results to threat assumptions

9. Mission Level Survivability • Vignettes Accomplished: percentage of vignettes where mission can be accomplished considering survivability constraints • Threat Shot Opportunities • Robustness Crew Survivability Engagement Level Survivability • Threat System Pk Envelopes • Robustness • Expected # casualties given a hit Vulnerability Engagement Level Susceptibility • Threat Envelopes (with and w/o CM) • F-Pole, A-Pole, E-Pole • Aircraft Pk/h (or damage given a hit or an intercept) Principal Survivability Metrics

10. Pk, OT&E, M&S and LFT&E(for Survivability Assessment) PK/E = PA/E*PT/A*PL/T*PI/L*PF/I*PH/F*PK/H OT&E M&S LFT&E E = Engagement A = Acquisition T = Track L = Launch I = Intercept F = Fuzing H = Hit K = Kill

11. Characteristics of System under Test ISA Process Create Vignettes Applicable to Program Generic Vignettes Program Documents (Example: TEMP, ORD) Data Collected from DT&E, LFT&E (to date and expected) Develop Test Plans Assess M&S Needs Data Collected from OT&E (to date and expected) Select M&S Suite Best-Suited for Program Library of M&S (incl. Credibility Information) Conduct T&E Execute and Collect data from M&S Suite Existing Vulnerability Data Collect Data from OT&E Collect Data from LFT&E Existing Susceptibility Data Conduct Analysis Assess and combine M&S and T&E Results Process Guidance and Constraints Survivability Metrics

12. Mission and System Level Test System Vignettes & Survivability Metrics Test Via Mission Operational Test Force Development/ Experimentation Tasks (UJTL) Interoperability Testing Interactions Actions Subsystem Test ANALYZE SYNTHESIZE Components Component Testing Technology Tech Demo Underlying Figure taken from ITEA Journal, Sept/Oct 2003, “The Test and Evaluation Master Plan: An Operational View”. Daly, Miller, Seglie & Wauer

13. Some Known Deficiencies in ISA Process • General Issues • Model linkages • M&S Validation data availability, coordination with TEMP • Mixed force level assessment (T&E and M&S) • Modeling and Simulation Deficiencies • DECM/IRCM effectiveness • Near field signatures, threat fuze performance • Fire and Explosion • Human performance • C4ISR • Test Range Deficiencies • Missile Miss Distance Measurement • Threat System Variability – system to system variations • Signal Density – not representative on ranges • Semi-Active systems – can’t get realistic test data on ECM effectiveness, etc. • Number of platforms, threats in test, test range size – can’t fully test integrated system • Completeness & fidelity of OT&E data – not generally usable for M&S validation, sometimes not sufficient for test issue resolution

14. ISA Demonstration • Demonstrate ISA process for an actual acquisition program • First candidate: Multi-Mission Maritime Aircraft (MMA) • FY04 Plans: • Develop focused ISA plan for MMA • Develop and coordinate checklist, metrics and vignettes with program office, LFT&E and OT&E agencies • Begin developing database requirements to support ISA assessment

15. Status & Recommendations • Draft ISA process developed • Identified checklist, metrics, models, test data needed • Notional examples worked (Transport, TACAIR, Helicopter) • Identified deficiencies in process, models and data • Draft report reviewed by Tri-Service committee • Developing detailed plans for process improvements • Fleshing out detailed ISA requirements for test data and M&S • Initiating test range capabilities assessment against those requirements • Correlated with ongoing range capabilities improvements • Provides roadmap for both M&S and T&E investment • Recommendations: • Complete and execute ISA process improvement plan • Coordinate with CTEIP and JASPO funding processes • Investigate POM plus-up opportunities • Investigate Training Range capabilities and limitations to support ISA • Develop detailed procedures manuals to supplement TEMP • ISA Demonstration

16. Backups

17. Links in the Threat Kill Chain ISA Metrics Potential Survivability Enhancement Features Along the Kill Chain Mission Survivability MissionsAccomplished; robustness All features combine to support mission level survivability Threat Suppression Threat Shot opportunities; situational awareness (number, timeliness and accuracy of threats detected) Tactics, Precision Guided Munitions, mission planning system, low signatures, fighter escort, anti-radiation missiles, self defense weapons Detection Avoidance Threat Detection & Acquisition Envelopes Standoff Weapons, Night-time capability, on-board electronic attack (EA) systems, stand-off EA, low signatures, good target acquisition, terrain following (NOE flight), situational awareness, chaff, threat warning, tactics, mission planning system Engagement Avoidance Threat Tracking envelopes; F-Pole, A-Pole, E-Pole; ECM effectiveness Standoff Weapons, onboard EA, off-board EA, low signatures, good target acquisition, situational awareness, chaff and flares, threat warning, speed and altitude, mission planning system Threat or Hit avoidance Threat Intercept Envelopes; ECM/IRCM effectiveness On-board EA, low signatures, chaff and flares, threat warning, speed and altitude, maneuverability, agility (last ditch maneuver) Threat or hit tolerance Threat system Pk envelopes; Aircraft Pk/h; Component Pk/h; VA; list of vulnerable components; Expected # casualties given a hit; hit locations on aircraft Fire/explosion protection, self-repairing flight controls, redundant and separated hydraulics, multiple engines, no fuel adjacent to air inlets, hydrodynamic ram protection, nonflammable hydraulic fluid, rugged structure, armor Metrics and the Checklist