1 / 25

OA Data Management An Overview of the Current Status of Data Management in Dstl

This paper provides an overview of the current status of data management in Dstl, including data quality score, collection and application of Exercise C2 data, and data management principles.

ojeda
Download Presentation

OA Data Management An Overview of the Current Status of Data Management in Dstl

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OA Data ManagementAn Overview of the Current Status of Data Management in Dstl Alan Robinson / Mark Taylor / Terry Hooper Dstl

  2. Dstl Data Management - ISMOR 20 • Current Status - General Alan Robinson/Terry Hooper - Land OA Toolset Mark Taylor • Data Quality Score Pete Bailey • Collection & Application of Exercise C2 Data Mark Ashforth • Data Management Principles: UK Perception of MORS W/Shop Mark Taylor/Terry Hooper Geoff Sherwood(Adaptive Ltd)

  3. What do we mean by data management? • Maintaining a body of required data for use by analysts • Facilitating the re-use of data where appropriate • Providing an audit trail - source, context and validation • Ensuring consistency and coherence across applications • Identifying need for data-collection or -generation • Establishing common standards for provision/use -where appropriate, encourage common tools & methods

  4. Why do we need effective data management ? • Effective data management will: • Improve the efficiency and validity of study work inter alia leading to reduced study costs. • Maintain consistency of data across studies and thus enhance coherence between them. • Through improved credibility, increase customer confidence in the data used and in the analysis that uses them.

  5. Why is it difficult? - (1) • Value of data management not well appreciated • Tendency to “muddle through” data difficulties • Failure to recognise potential spend-to-save • No sound assessment of the status quo against which to assess improvements and likely value for money • Co-ordination across data management initiatives not always achieved • Partly due to lack of customer co-ordination • Partly due to labelling - not all relevant work is labelled as data management; data related activity often occurs as part of other projects

  6. Why is it difficult? - (2) • Managing data effectively requires an understanding of context • OA pulls together military, technical and other inputs often without the benefit of common terminology and data standards • There are usually interdependencies between data items • OA models and ‘softer’ methods have grown in complexity and been brought to bear on (even) more challenging problems • e.g. C2, OOTW and Strategic Analysis.

  7. Why is it difficult? - (3) • Toolset support has often not been as effective as desired: • Often tightly coupled to a target model • Some false starts • Advances that should enable more generic toolset support only recently becoming available and mature • e.g. XML for data transformation

  8. Progress to date • Progress has been made with necessary process, data sources and toolset support, for example: • Dstl data management strategy and associated handbook exists • GENeric Information Environment (GENIE) toolset under ARP15 • Network of data custodians captured via an OA Data Directory …however, improvements in data management have not been as significant as anticipated

  9. Where next?(1) • Managing “data with context” (information ?) has proven harder than originally expected • Progress requires both: • “Research” activity - to address research, development and transition-to-study-use issues • “Management” activity - to ensure appropriate processes and culture are in place

  10. Where next? (2) • “Research” activity - • Needs customer support and funding • to address research, development and transition-to-study-use issues • “Management” activity - • Needs Dstl action with appropriate customer support • to ensure appropriate processes and culture are in place • Neither strand alone will suffice

  11. Recommended areas to be tackled • M - Develop a network of responsible data co-ordinators • M - Overhaul and re-issue Data Management Handbook • R - Develop appropriate toolset support, using generic methods wherever practicable • R - Improve methods for setting data in context • R - Investigate and, if appropriate, adopt data standards • M/R - Establish method of assessing benefit of data management • M - Improve co-ordination in both customer and Dstl

  12. ARP15 Data Management • Funded by ARP15 over the past 3-4 years • Previous work has focused on the development of tools • FY03/04 programme on focused on: • Management of scenario data • Continued rollout of existing tools • Implementation of a data management team within LSD • Continuation of international links • Support to wider Dstl and MoD data management activities

  13. File Input Exchange Data Manoeuvre Threat Manual Formats Tools Sub System Data (e.g. ARP 07 Database) XML System Capability Static Models Model Database Model Model Post Data System Data (GDMS) Data Input Runs & Run ) (GDMS File Checking Data Output Analysis Files Files Terrain Chunk Terrain Data (DGIA) SAG File Model Run Management System ORBAT Files & SID Terrain Model Boundary Data ORBAT SAG & JCD Process JCD (SID) File ORBAT & Deployment Files Current GENIE Tools Study data Input Data Key Manoeuvre Compendium of Scenarios Data Transforms ARP15 Data Management Framework

  14. Scenario Data • ARP15 Manoeuvre Compendium of Scenarios: • Contains scenario information at Formation level and below • Has links to the electronic SAG book and existing JCD scenario descriptions • Existing lower level scenario documentation grouped by SAG setting and study • Data accessed via a web page on the Dstl secret network • Scenario documents can be download from the compendium in electronic format

  15. Toolset Development • Work has been undertaken within ARP15 over the past 3-4 years to provide a set of generic tools to support data management for individual models and across capability areas • Known as the GENeric Information Environment (GENIE) • Initially focused on the management of model input data • Can be extended to cover other areas, e.g. model run management and post run analysis • GENIE is transitioning from development into routine use

  16. GENIE • The current GENIE toolset consists of three applications: • Generic Database Management System (GDMS) • Stores static data in a modified Oracle database • COTS data transformation tools • Mercator and XMLSpy • Scenario Interactive Display (SID) • ORBAT definition and deployment on to the terrain • Model input data entry • Generation of model terrain databases • Replay of model runs

  17. GDMS Overview • Built on Oracle COTS database software • Multi-user • Designed to store large volumes of data • Data defined objects and templates • Forms based front end • Primary role is to support capability databases rather than model specific data

  18. Data Transformation • Data transformation is needed to: • Change the format of data • Aggregate or disaggregate data • Data is transformed using: • Mercator COTS product • Bespoke tools based on Excel, VBA, and high level programming languages e.g. C++ • Experiments with XML have shown its potential, despite limited mathematical capabilities

  19. SID Functionality • Provides the following functionality: • Model data dictionary specification • ORBAT definition and deployment of units onto terrain • Cut and paste functionality to assist with ORBAT set-up • Propagation of changes to data dictionary to extant ORBAT files • Generation of terrain data for models • Generation of Line-of-sight statistics from digital terrain databases • Replay of model runs

  20. Data dictionary / ORBAT generation

  21. Terrain Generation • Can be generated at user defined resolution • DTED, DFAD, DCW and VMAP level 0 data sources can be used • Culture / features can be painted on to terrain and saved

  22. Unit deployment

  23. Replay • Requires unit ORBAT to be set-up in SID • Potentially any model output can be replayed • User can define colours and details viewed

  24. SID current status • Functionality available in version 2 has been improved to cover core areas noted • Tool has been thoroughly beta tested • User guide has been completed • Tool is ready for release • Users: • Tool is being used actively by WISE, FCM and CLASS teams • CLASS will soon start using SID for static data entry

  25. Conclusions • Data management within ARP15 has proved a far harder problem to solve than originally expected • The real need is to manage information, i.e. data and its associated context • Results in increased technical and cultural complexity • Data management tools only provide part of the solution • Requirement is to define a workable data management process that operates in conjunction with the tools • Will require support from MoD and Dstl stakeholders, ARP15 and Dstl LSD

More Related