1 / 16

A Prescriptive Adaptive Test Framework (PATFrame) for Unmanned and Autonomous Systems: A Collaboration Between MIT, U

Massachusetts Institute of Technology . A Prescriptive Adaptive Test Framework (PATFrame) for Unmanned and Autonomous Systems: A Collaboration Between MIT, USC, UT Arlington and Softstar Systems. Dr. Ricardo Valerdi Massachusetts Institute of Technology March 10 ,2010. Outline.

thurman
Download Presentation

A Prescriptive Adaptive Test Framework (PATFrame) for Unmanned and Autonomous Systems: A Collaboration Between MIT, U

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Massachusetts Institute of Technology A Prescriptive Adaptive Test Framework (PATFrame) for Unmanned and Autonomous Systems: A Collaboration Between MIT, USC, UT Arlington and Softstar Systems Dr. Ricardo Valerdi Massachusetts Institute of Technology March 10 ,2010

  2. Outline • The Challenge • PATFrame team • PATFrame objectives & scope • PATFrame features • Next steps “Anything that gives us new knowledge gives us an opportunity to be more rational” - Herb Simon

  3. Sponsors Transition Partners

  4. PATFrame kickoff meeting Fort Hood, TX - Aug 2009

  5. The Challenge: Science Fiction to Reality “You will be trying to apply international law written for the Second World War to Star Trek technology.” Singer, P. W., Wired For War: The Robotics Revolution and Conflict in the 21st Century (Penguin, 2009)

  6. PATFrame Objective To provide a decision support tool encompassing a prescriptive and adaptive framework for UAS SoS Testing • PATFrame will be implemented using a software dashboard that will enable improved decision making for the UAS T&E community • Focused on addressing BAA topics TTE-6 Prescribed System of Systems Environments and MA-6 Adaptive Architectural Frameworks • Three University team (MIT-USC-UTA) draws from experts in test & evaluation, decision theory, systems engineering, software architectures, robotics and modeling http://mit.edu/patframe

  7. Prescriptive Adaptive Test Framework Test Strategy/ Test Infrastructure Prescriptive Adaptive System under test

  8. Time Scale for Testing Decisions PATFrame Scope Test Development (i.e., design for testability) Long Term Planning Decisions/ investments Test Planning (in SoS environment) Data Collection, analysis & reprioritization Test Execution (real time) Days Months Hours Minutes Years

  9. Net-centricity of the environment net-centric focus Testing a system in a SoS environment Difficulty SoS Testing a SoS in SoS environment Ultimate Goal DARPA Urban Grand Challenge Use case (UAS in SoS) UAST focus PATFrame Initial Focus: Testing Autonomous System in SoS environment Complexity of the system under test none Human SoS S AI Autonomy of system under test Testing SoS

  10. Normative Prescriptive “Successful SoS Test” = f(metricA, metricB, etc.) (MetricA) limit Descriptive Normative (best) Descriptive (SoP) Descriptive (actual) Potential (new test A) test A contribution to state of the practice “success” … … MetricA MetricB MetricC MetricN Prescribed System of Systems Environment Goal: Synthetic framework for SoS testing at single and multi-program level Goal: Construct Theoretical Best “SoS” Test Metric set, “best” levels Metrics, state of the practice levels Goal: Capture Actual SoS Test Actual SoS tests include metricA’, metricC, etc.

  11. Integrated Test Management

  12. Real Options as Prescriptive and Adaptive Framework for SoS Testing • Use case: • Question: what to test? (i.e. what SoS test scenarios to prescribe?) • Inputs: models of autonomous/net-centric system of systems, major uncertainties • Outputs: enablers and types of real options for responding to uncertainties as candidate test scenarios (i.e. identification of how SoS can adapt) Identification of Real Options: Objective: Maintain Vehicle1Vehicle2 comm. Uncertainty: proximity of vehicles 1 and 2 1. Real option to adjust comm. range using flexible range comm. systems on vehicles1, 2 2. Real option to use Vehicle3 as relay Joint/ SoS Model: coupled dependency structure matrix Vehicle1 (Army) Ontology Vehicle2 (Navy) 3 2 Uncertainties Vehicle3 (Air Force) 1 Mission objectives

  13. Testing to Reduce SoS Risks vs. the Risks of Performing Tests on an SoS

  14. Example PATFrame Tool Concept Question: When am I Done testing? Technology: Defect estimation model Trade Quality for Delivery Schedule Inputs: Defects discovered Outputs: Defects remaining, cost to quit, cost to continue 14

  15. When am I done testing?

  16. PATFrame Knowledge Base Analysis Techniques Primary Inputs to PATFrame Primary Outputs for Test and Evaluation Models of the system under test • How should my test • strategy change? Reasoning Engine Model of the SoS environment • What realizable options • are available? Mission needs/goals Plan Ontology • Which test do I run • and in what order? Test Requirements Analyze and Design Execute • When am I done • Testing? • Available resources • and time Evaluate Risk & Uncertainty Analysis (leading indicators) Process Analysis (System Dynamics)

More Related