1 / 28

Test & Evaluation/Science & Technology Program Unmanned and Autonomous Systems Test (UAST)

Test & Evaluation/Science & Technology Program Unmanned and Autonomous Systems Test (UAST). FY2009 Program Execution Review. Objectives. Project Status UAST Accomplishments UAST Lessons Learned Summary. UAST Technology Overview. UAS Layered Architecture. T&E capability shortfalls

Mia_John
Download Presentation

Test & Evaluation/Science & Technology Program Unmanned and Autonomous Systems Test (UAST)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Test & Evaluation/Science & Technology Program Unmanned and Autonomous Systems Test(UAST) FY2009 Program Execution Review

  2. Objectives Project Status • UAST Accomplishments • UAST Lessons Learned • Summary

  3. UAST Technology Overview UAS Layered Architecture • T&E capability shortfalls • Predicting UAS Intelligence and Behavior • Emulating Mission & Environmental Complexity • Assessing UAS Effects and Capabilities • UAST Protocols and Design • UAS Test Bed and Environment • UAST Reference Data (Ground Truth) • UAST Tools and Techniques • Major technology challenges (in the S&T solution development) • Develop algorithms for predictive behavior by integrating genetic algorithm, probability theory, and autonomous behavior theory • Construct models to analyze UAS decision operations in complex and uncertain environment • Develop methodologies and metrics to evaluate distributive control and situational awareness • Develop test protocols and algorithms to support on-board management system evaluation and UAS shutdown • Develop realistic test bed components, and environments to support UAS evaluation approaches to unfolding missions via range assessments • Compile ‘truth data’ to support range safety, and protocols development Autonomy Knowledge Cross-layer Mgmt and Control Interaction Net-Centric Comms (GIG) Platforms Sensors Weapons

  4. How does the focus area prioritize the technology challenges (state your rationale)? Mapping of T&E needs to POR or NPOR as defined by: UAST WG DoD Roadmaps SPO/ Project Managers By technology having broad applications across UAS Domains UAST Technology Overview (cont.) UAS Layered Architecture Autonomy Knowledge Cross-layer Mgmt and Control Interaction Net-Centric Comms (GIG) Platforms Sensors Weapons UAS Architecture Layer Emulations Autonomy Mission and Behaviors Knowledge Injection Knowledge Cross-layer Mgmt and Control Metrics and Analysis Collaboration and Actuation Interaction Virtual, Constructive, And Live (mixed) Net-Centric Comms (GIG) Platforms Sensors Weapons Environment (Physical, Tactical, Social, etc.) Terrain, Weather, Comms, Threats, Friendlies, Neutrals Air, Space, Surface, Subsurface

  5. NIST Performance Decomposition Model As opposed to componentized UAS view presented in roadmap Another alternative would be a test framework over ALFUS Predict Roadmap leverage of accepted reference models for testing UAS Assess Emulate

  6. Operational UAST Challenge

  7. Autonomy-Base UAST Challenge

  8. Key Future Events & Dates • Start/Finish : October 2008/September 2010 • PDR: July 2009 • Phase 2 Component Demo: September 2009 • Phase 2 Scientific Report: September 2009 • CDR: January 2010 • Final Integrated CAST Demo: August 2010 • Final CAST Scientific Report: September 2010 • Final CAST Transition Report: September 2010 Project Description The overall CAST methodology learns each mission’s goals, learns vehicle-specific characteristics and behaviors, and identifies vehicle behavioral anomalies. In addition, a self-organizing knowledge representation system will organize and categorize the information in a way that permits the tester to test complex emergent behaviors and anomalous behaviors to be differentiated and evaluated during highly difficult missions and mission failure modes. • Transition Partners & Dates • Army Research Lab-BCT (FCS) Mission Based Applications. 4Q FY10 • Army Advanced Technology Directorate (AATD), TBD • Deliverables • CAST integration strategy report • Stand-alone toolkit for UAS autonomous behavior testing • Final Project report Cognitive Autonomous System Tester (CAST) Project 1 of 13

  9. Key Future Events & Dates • Start/Finish : January 2008/May 2010 • Phase 2 Documentation: September 2009 • Collaborate with transition partner: December 2009 • Define User Cases: January 2010 • Incorporate Selected assets Demo: February 2010 • Effectiveness of interaction and multiple assets report April 2010 • Final Integrated report: May 2010 Project Description HFCMA is a Live/Virtual/Constructive T&E framework that merges Live and Constructive assets, including heterogeneous collaborative UxVs with the ability to mirror One to Many Live UxV’s performance in the V/C simulation. T&E Use Case is a) multiple vehicles interacting, b) each with collateral systems and c) a test system that must coherently stimulate the vehicles under test and capture limits of UxVs whole system performance and integrity. HFCMA is concerned with testing the dynamic limits of autonomy • Transition Partners & Dates • Incorporate use cases for air, ground & sea • Verify adequacy with transition partner • ATEC-BCT(FCS) Robotic convoy 1Q/FY10 • NAVAIR/452 FTS-Scan Eagle/Predator 2Q/FY10 • NUWC-Undersea environments TBD • Deliverables • LVC framework for test data recovery and analysis • Delivery of prototype test framework • Framework training manual • Final project report High-Fidelity Communications, Modeling & Analysis (HFCMA) Project 2 of 13

  10. Remote Embedded System Test (REST) Project Description REST is the integrated set of technologies that replaces a wired instrumentation system with a wireless, self powered, self organizing, self healing network of sensor nodes designed to be manufactured (embedded and assembled) into the platform structure and provide continuous test and evaluation capability as well as integrate into the platform Integrated System Health Manager. REST is designed to provide UAST capability for multiple platform types and DoD services. • Deliverables • Prototype TRL 6 system components • Flight and Ground Demonstration • Final Report and White Papers • Key Future Events & Dates • Start/Finish : October 2007/December 2009 • Field Demonstration: August 2009 • Final Report: October 2009 • Conduct Field Test November, 2009 • Final Report: December, 2009 • Transition Partners & Dates • Advanced Composite Cargo Aircraft (ACCA) Program, Phase III FY09 (REST TRL 6 Baseline Technology) – Installed and evaluated in ACCA Fuselage Fatigue Test Article (Mar CY2010) • Vulture (DARPA), In-Work Project 3 of 13

  11. Key Future Events & Dates • Start/Finish : August 2008/August 2009 • Transceiver Completion Report: August 2009 • Transceiver link Demonstration: October 2009 Project Description Flexible multi-band command and control communications system supports test data capture and command/control applications during complex multi-agent (UAS) test scenarios. Effectively implements a software-customizable transceiver system that is frequency agile with modulation and protocol independence.SCMT encompasses four transmitters and four receivers in a single unit the size of a commodity disk drive programmable for any frequency band between 1 MHz and 6 GHz. • Transition Partners & Dates • Phase 1 completion Oct 2009. Must coordinate with SET to determine if project will continue • Test community interest: 452nd Test Wing, NAVAIR, ATEC • Deliverables • Project will provide a 4-channel SCMT demonstration prototype • Set of capability demonstrations for core functionality • Aeronautical TM modulation formats Software Configurable Multi-Channel Transceiver (SCMT) Project 4 of 13

  12. Key Future Events & Dates • Start/Finish : October 2007/September 2009 • System Design Review: June 2008 • Field Demonstration: August 2009 • Final Report: October 2009 Project Description Effective evaluation of a UAS performance depends on ground truth about the world, the vehicle and what the vehicle senses. MTAS is designed to provide vehicle localization data utilizing small self-contained, and inexpensive RF micro-beacons affixed to test vehicles. Portable ground stations receive the signal and calculate range to each test vehicle.This supports multiple autonomous systems, low latency, sub-meter positioning performance, and a full performance operation in GPS-denied environments. • Transition Partners & Dates • Brigade Combat Team Modernization (BCTM) Experiment (RCX), 4Q/FY09 • Deliverables • Three prototype beacons • Full scale system demonstration at Ft. Devens airfield • Final project report Micro-beacon Tracking of Autonomous Systems (MTAS) Project 5 of 13

  13. Key Future Events & Dates • Start/Finish : April 2008/November, 2010 • End of Phase 2 Demo: February, 2010 • Start of Phase 3: February, 2010 • End of Phase 3: November, 2010 Project Description. VATT is a tool used for autonomous vehicle simulation, teaming, and decision making analysis. It aims to develop a VATT configuration for T&E applications, that can predict how modifications to external stimuli will produce changes within the UAS autonomy logic processes. VATT environment strives to seamlessly integrate simulation subsystems with real UAS hardware subsystems. Simulated systems could replicate multiple platforms for the development of mutli-platform behaviors thus supporting repeatable events and detailed system/ event logging. • Transition Partners & Dates • RTTC; John Hardison- 4 Q FY10 • Deliverables • Requirements Document • VATT-T Software Toolkit • Final Report Virtual Autonomous Teaming Tool(VATT) Project 6 of 13

  14. Key Future Events & Dates • April 2008/December 2010: Start/Finish • Sept 27-28 : Final Large Prototype Static Test • Oct 15-16 : Final Small Prototype Static Test • Nov 5-6 : Final Large Prototype Dynamic Test • Nov 26-27: Final Small Prototype Dynamic Test • Dec 10-11: Final Demo Phase II Project Description High precision site data is an essential tool in deriving ground truth. Comparison not only of the vehicle’s position with actual location but with what the vehicle senses in its surroundings. This S&T project is based upon analytical studies and laboratory studies to physically validate theoretical foundations using software and standards for the implementation of multiresolution models for unmanned autonomous systems testing. The analysis, models, software and standards will be evaluated through both hardware emulation and experimental evaluation. • Transition Partners & Dates • Army Research Laboratory, 4 Q/FY10 • Air Force Research Laboratory, 4 Q/FY10 • Other dates of interest: • ARL Test Collaboration TBD LIVEFIRE Program • AFRL/NASA Phase II SBIR – TBD Helicopter flight demonstration • Deliverables • Software module libraries modified for hardware deployment. • Ground / Air testing of laptop payload • Detail design specifications of Prototype FPGA-based I/O Board. Standards, Software and Hardware for Multi-Resolution Maps/Models (MRMM) Project 7 of 13

  15. Key Future Events & Dates • Start/Finish : September 2008/September 2010 • CDR: April 2008 • Phase 1 Demo: September 2008 • Phase 2 Demo: August 2009 • Final Report and User Documentation Project Description IABF will allow for systematic and structured T&E of technologies in the cognitive/social domain using a combination of simulation, hardware in the loop testing and live testing. Testers will quickly and efficiently be able to develop test plans and validate UAS Team performance with respect to: • C2 navigationapproaches utilizing NLOS techniques • Collision avoidance • UAS Team coordination and performance • Fault tolerance under various failure modes and bandwidth constraints • Transition Partners & Dates • Picatinny Arsenal, ARDEC,1Q FY10 • Naval Research Lab (NRL),4QFY09 • OneTEss/PEOSTRI/OTC, 4QFY09 • Deliverables • DCTF Software and libraries • Phase I : Indoor demonstration of Test-framework • Phase II: Outdoor advanced Technology Demo • Final report and User documentation An Integrated Agent-based Framework for Simulated, Virtual and Live Testing of Teams of Unmanned Vehicles (IABF) Project 8 of 13

  16. Key Future Events & Dates • Start/Finish: January 2009/Sep 2009 • LVC & RWMS Demonstration: August 2009 • Phase 2: Prototype & Documentation: Sept. 2009 Project Description Ground truth measurement infrastructure to support UAS testing and vehicle safety analysis. The measure system consists of components that can provide an alternative source of data to compare with those organic to the system under test. Three levels of modularity, namely, i) physical interface (face plate), ii) hardware stack, and iii) computational fabric (FPGA based) are employed to provide maximum flexibility • Transition Partners & Dates • ATEC White Sands Missile Range, 1QFY10 • NAVAIR, 1QFY10 • 412 FTS EDWARDS, 1QFY10 • Deliverables • RWMS prototype (HW and SW) • Documentation of the miniaturized RWMS design RWMS Demo in 2 or more UAS test case Scenarios • Final report and analysis of demonstrated capability Reconfigurable Wireless Measurement System (RWMS) Project 9 of 13

  17. Key Future Events & Dates • Start/Finish: October 2007/December 2009 • Acquisition of telemetry source: June 2008 • Delivery of Second Telemetry Source: Sept. 2008 • Initial Demonstration: December 2008 • FCCS workbench demonstration: August 2009 • Delivery of Final Documentation: December 2009 Project Description FCCS is a user facing application framework that provides C2 capabilities, its primary functions are to reduce the amount of time to display meaningful data to the user and improve the C2/HRI functionality of the targeted robotic device. When a new UAS is presented for testing, The C2 system must be able to be rapidly adapted to the unique qualities of the new UAS. The C2 system must also have a standardized, well-documented framework so that UAS developers can prepare their UAS for integration with the C2 system before arriving at the test facility. • Transition Partners & Dates • Global Observer 412 FTS EDWARDS, 1QFY10 • Deliverables • Prototype Demonstration • FCCS Documentation • FCCS Final System Flexible Command and Control Systems (FCCS) Project 10 of 13

  18. Key Future Events & Dates • Start/Finish : October 2009/September 2012 • PDR: September 2010 • Event Workshop 1&2: September 2010 • CDR: March 2011 • Event Workshop 3: March 2011 • Phase2 :Progress Report Tool: September 2011 • Event Workshop 4: September 2011 • Final Demonstration: September 2012 • Final Demonstration: September 2012 • Final out brief & Deliverables: September 2012 Project Description PATFrame is a decision support tool encompassing a prescriptive and adaptive framework for UAS SoS Testing. PATFrame will be implemented using a software dashboard that will enable improved decision making for the UAS T&E community. The technology addresses the Modeling & Architecture and Testbeds & Test Environments areas in the UAST capability framework. • Transition Partners & Dates • U.S. Army Operational Test Command (USAOTC), Ft. Hood, TBD • Naval Air System Command (NAVAIR), Patuxent River, MD, TBD • Air Force: 303rd Aeronautical Systems Wing, Wright-Patterson Air Force Base, OH, TBD • Deliverables • Tool delivery (Software) • Final demonstration • Final Documentation A Prescriptive and Adaptive Testing Framework (PATFrame) Project 11 of 13

  19. Key Future Events & Dates • Start/Finish : October 2009/October 2012 • Demo underwater motion capture system: Oct 2010 • Demo Model-based simulation and hardware prototype: October 2011 • Phase 2 Scientific Report: October 2011 • Prototype deployment and demonstration: 2012 Project Description SCULL applies dynamic systems theory to design reduced order models of cooperative UUV navigation, sampling, performance, and control. UUV technology includes operations in remote/inaccessible, and dynamic environments; autonomous trajectory generation, tracking, and coordination. The value added to UUV testers is (1) tests to guide UUV development and deployment; (2) help to define T&E for cooperating underwater vehicles; and (3) provide T&E with a capability for fielding autonomous systems that are suitable, effective, and survivable. • Transition Partners & Dates • Naval Undersea Warfare Center (NUWC), 1Q FY12 • Deliverables • Interim report on reduced-order modeling of cooperative behavior • Interim & final reports on model-based simulation framework • Prototype micro-UUV platform and software Synthetic Collective Unmanned Underwater Laboratory (SCUUL) Project 12 of 13

  20. Key Future Events & Dates • Start/Finish : October 2008/December 2009 • Progress report of current tasks: December 2008 • Ontology/Taxonomy task begun: January 2009 • Protocols Document: December 2009 • Software solution for Ontology researched and identified: December 2009 Project Description Proposed technology is a developmental support tool that focuses on a collection of entities for UAS testing. This support tool is known as an Ontology/Taxonomy application containing knowledge about existing UASs. Real time positioning of equipment to monitor UAS experiments via the internet for T&E observations occurring within the UAST Enclave. The ontology/taxonomy will be implemented as a decision support system to improve UAS T&E outcomes. Appropriate UAS choice for the mission & environment. • Transition Partners & Dates • UAST WG/ UAST Prototype Testers, 4Q FY10 • Deliverables • Development of Test Protocols • Evaluation tool • Ontology/Taxonomy working application • Final Report UAST Technical Topic Areas = Framework M&S for UAST Aspects & Protocols UAST Analytics Modeling and Architecture Test Beds & Test Environments JPL B SNL LU Lab Ft. Hood Picatinny WSMR Obstacles UAST Network UAS ( ”, t ”,”, x”, y”, z”) Ft. Bliss A ( ’, t’, x’, y’, z’) ( ,t, x, y, z) Project 13 of 13

  21. UAST FY09 Achievements

  22. UAST FY09 Achievements (cont.)

  23. UAST FY 09 Lessons Learned • UAST Working Group MUST shift focus from 'marveling about UxS's' to 'T&E'ing UxS's.' We must devise a UAST framework so that the S&T that will enable various 'paths' through the framework can be highlighted and  justified then the ROI of S&T projects can  be measured.  • We can move from a focus on a) measuring the autonomy algorithm in the payload to b) measuring the autonomy of the Whole System and doing so sufficiently to Predict, with stated Confidence, the anticipated Suitability, Safety, Effectiveness and Survivability of the whole system.  "How can you 'prove' that your systems design will solve the customer's problem before you build and prove that design?"  • The UAST process must be synchronized with PMO's view of the world and the TRMC process. Drop 'ConOps' and talk in terms of Use Cases and Use Case Portfolios.  Then S&T projects must state Objective in terms of Use Case hot spots and a) S&T intent regarding ameliorating the hot spots and b) requirements allocated to other of TRMC's S&T activities.   • Use Cases must state the T&E implications of the whole UxS. T&E implications are objectives and concepts as stated at the Effects --> Capabilities -->Algorithm/Timeline level (not System Shall statements) across development, prototype, IOC, production

  24. Accelerating UAS Deploy through Test

  25. Safety in the Battlespace

  26. Three Phases of Startups • Glimpse of “Market” • Learning enough to match market with technologies • Create value proposition for garnering ROI • Variety • Extent • Ambiguity of Operational Necessity; Fast Tracking; Need for Testing for Safe, Suitable, Effective, and Survivable UAS We are currently finishing Phase II

  27. Three Questions • Are we progressing fast enough to help the Warfighters? • Prototypes, Demos, Beta-Testing • How can we accelerate S&T activity for UAST? • Use cases • Connections to UAS programs: workshops, outreach, conferences, web site, magazine, flyer • Other? AUVSI, Robotic Rodeo, PERMIS, NIST, SAE, INCOSE, IEEE, NDIA, UARCs, Academia • These rules for triage? • Must address a T&E need • Must require S&T • Should have a big payoff • Should have wide applicability • Should have a transition partner

  28. Summary • UAST Working Group • MUST devise a UAST ROI framework • Use-case mapping of test needs for safe, suitable, survivable and effective UAS • Technology forecasting • CTEIP Mitigation • Solicit needs from Warfighters, Testers, Developers, PMs • Transition partner development • How do we accelerate S&T investments into T&E capability? • UAST overarching role • Filling in the gaps for testing UAS that are being deployed using rapid acquisition practices • Focus on autonomy T&E • Focus on safety of UAS • Stakeholder collaboration (COI-UAST) • Use Cases must state the T&E implications of the whole UxS • T&E implications are objectives and concepts as stated at the • Effects --> Capabilities -->Algorithm/Timeline level (not System Shall statements) across development, prototype, IOC, production

More Related