1 / 15

Measuring the Quality of Decisionmaking and Planning

Measuring the Quality of Decisionmaking and Planning. Framed in the Context of IBC Experimentation February 9, 2007 Evidence Based Research, Inc. Today’s Ground Rules. IBC provides the framework for the discussion on measuring decisionmaking. This is not an IBC brief.

donnan
Download Presentation

Measuring the Quality of Decisionmaking and Planning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring the Quality of Decisionmaking and Planning Framed in the Context of IBC Experimentation February 9, 2007 Evidence Based Research, Inc.

  2. Today’s Ground Rules • IBC provides the framework for the discussion on measuring decisionmaking. • This is not an IBC brief. • The briefer does not have the answers! • The purpose is to explore how one measures the quality of decisionmaking in complex environments – this is a discussion session.

  3. Background Regarding IBC Focus: • Visualization, operational assessment, and planning tools for EBO. Multiphased effort: • Phase 0: Proof of concept using ad hoc suite of tools. • Phase I: Demonstrated improvement potential of federated tool sets developed by two contractors. • Phase II: Evolve tool set to expanded capabilities for transition to JFCOM & COCOMS.

  4. Objectives of Phase II Experimentation • Establish the operational utility of the tools. • Learn which tools can be helpful under what circumstances and for what purposes. • Explore novel approaches throughout the spiral development process. • Measure progress in increasing readiness of the technology. • Facilitate early transition to operational users.

  5. Challenges • Complex operational problem: • Wide range of situations, • Multiple knowledge domains, and • Poorly understood relationships. • Major technical challenges: • Variation in maturity of underlying models and tools, • Limitations in availability of relevant data, and • Composability of federated capability from continually expanding set of tools. • Transition hurdles: • Operational risk due to high potential for limitations in fidelity and possible misapplication.

  6. Option Exploration Tool (1) • Comprehensive family of models. • Action-to-effect and effect-to-action. • Generates the distribution of plausible outcomes. • Visualization: assist leaders to understand and act.

  7. Option Exploration Tool (2) Proposed Action Undesired Effect Desired Effect Desired Effect Modeling Paradigm Conceptual Models Spread Sheet Political/Economic Model Concept Map Statistical Models Social Network Differential Equations Influence Diagram Political/ Religious Model Causal Models Petri net Bayesian Network Social/ Culture Model Object Models Economic/Infrastructure Model Event-based Simulation Agent Based Simulation DIME Dimensions Social Information Model PMESII Dimensions Religious Model Military Model Enables “What-if” Analysis, Answers “Why” Distribution authorized only to US Government Agencies and their contractors involved in Phase 2 of the Integrated Battle Command Program

  8. Campaign Planning Tools (1) • Supports modular development of multiple lines of effort. • Automatically detects and displays interdependencies, assumptions, resources, actions, duration of effects, metrics, and next state. • Ability to modify plans, actions, interdependencies, models, and next states based on measured performance.

  9. Campaign Planning Tools (2) Political/ Strategy Security Strategy Interdependencies Assumptions Political-Economic Strategy Actions Next State Visualization Resources Measured Performance Allows the leader to visualize complex plans Distribution authorized only to US Government Agencies and their contractors involved in Phase 2 of the Integrated Battle Command Program

  10. Network Centric Operations

  11. Decisionmaking Quality • Two approaches: • Quality of the process: • How the decision was made, and • Elements in the process. • Quality of the decision: • Was it “right” or at least “not wrong”? • Did it have the desired effect?

  12. Quality of the Process • Correct sequence of actions. • Complete set of choices. • Assumes an observable process. • Repeatability. • Outcome is irrelevant.

  13. Quality of the Decision • Applicability to the solution space. • Compatibility with other SMEs. • Desired outcome/effects. • Outcome is relevant.

  14. Measuring Quality • The field needs a way to measure the quality of a product: • What does it mean to do good? • What does it mean to make a good decision? • A simple problem: • Has a correct answer. • A complex problem: • Has no right answer, • May have “not wrong” answers, and • Will have multiple perspectives.

  15. Discussion Time Thoughts?

More Related