270 likes | 608 Views
Dive into the data management life cycle strategies implemented by USGS in its Landscape Energy Program. Explore the centralized storage, metadata utilities, and innovative data access systems. Learn about the organization's information services and future directions.
E N D
A Data Management Life-Cycle By David Ferderer Project Chief Chris Skinner Contractor Greg Gunther Contractor dferdere@usgs.gov
Presentation Outline • USGS Landscape • Life-Cycle Model and Strategy • Component Descriptions (Skinner) • Demonstration (Gunther) • Conclusions and Future Directions
USGS Landscape - Energy Program • What We Do • Provides Science-Based Energy Assessments • Organization Issues • Regional Centers and Competitive Funding Process • Multiple Project Areas, Applications, Data Types, and Platforms • Information Issues • Technology and Data Explosion • Access, Delivery, and Archive Requirements • Diverse Client and Product Needs • Policy and Mandates
USGS Landscape - Central Energy Team • 125 Full and Part-Time Employees • Independent Thinkers and Researchers • Multiple Application Platforms • UNIX (ArcInfo 8, ArcView 3x, SDE 3, ORACLE 8, EarthVision, Seismic, PETROMOD) • PC/NT (ArcInfo 8, ArcView 3x, Geographix) • Centralized and Distributed Data Storage • 100mb Fast Ethernet Network
Central Energy Team “Information” Shift Information Services Data Management Project Life-Cycle GIS Integration
Life-Cycle Model and Strategy • Life-Cycle Model (Conceptual) • A Series of Processes and Utilities that Manage the Flow of Data to Information, Products, and Knowledge • Life-Cycle Implementation Strategy (Actual) • Processes are Translated into the Find, Get, Use, Deliver, and Maintain Strategy • Strategy Defines Tasks, Components, and Deliverables
Implementation Strategy • DM Finds Internal and External Data Resources • DM Gets the Data Organized, Documented, and Accessible to Team Projects • Projects Use the Data and Other Resources in Research • DM Assists Projects in Delivering Products to Public • DM Maintains the System and Upgrades Components
Strategy Components and Utilities (Internal USGS) Team Data Library Archive Library Inventory Database Metadata Utilities CD-ROM Templates Data Processing Utilities Hypermedia Publications Project Design Intranet Resources Find External Data and Information Get Data Organized Find Internal Data and Information (Archive and Reuse) Data Life-Cycle Maintain (Upgrades and Documentation) Deliver Data and Knowledge to Projects and the Public Use Data and Other Resources In Research Projects
Team Data Library Team Data Library • Centralized Storage • Team Data Resources (primarily spatial) • Theme and Sub-Theme Organization • Standardized • Naming Conventions • Directory Structure • Storage Formats (e00, shape, SDE) • Common Data Projection (geographic) • Metadata • Browse Graphics
Archive Library Team Archive Library • Offline Storage of Team Data Resources • Contains • Publications • USGS Digital Data Products (DLG, DEM, DOQ) • Team Archives • Standardized File Names and Directory Structure
Inventory Database Inventory Database • MS Access Database Tracking Team’s Data Holdings • Contains • 60 Information Fields (10 Required) in 21 Tables • 28 Fields Corresponding to FGDC Metadata Elements • Inventoried 4600 Datasets and 680 Archives (> 500 GB)
Inventory Database Inventory Database • Features • Tracks Multiple Types of Data (Spatial, Text, Graphic and Tabular) • Separately Tracks Archives, Publications, and Individual datasets • Automatic Loading and Editing Scripts • Serves as the Engine to DART…
DART • Data Access, Retrieval, and Tracking System • Easy Access to Team Data Resources via Web Browsers • Customized Search and Browse of Archives, Publications, and Datasets • Direct Data and Metadata Download to User’s Desktop • Object-Oriented Application • Java Server Pages on ServeletExec 3.1 • Stay Tuned for the Demonstration!
Metadata Utilities Metadata Utilities • Web-Based Metadata Entry and Creation System • Users Generate, Modify, and Save Compliant Metadata Output to the Desktop • Provides a Simplified and Comprehensive Online Help System • Contains • Links to Other Metadata Tools and Resources • Library of Metadata
Hypermedia Publications CD-ROM Templates Data Processing Utilities Project Design Other Data Management Products • Data Processing and Automation Utilities • Portal to ‘How-To’, AMLs, and FAQ Documents Residing in the Team and On the WWW • Project and Workspace Design Recommendations • Templates Promote Efficient Work-Flow, Data Organization, Archives, and Rapid Publication • CD-ROM Templates and Hypermedia Distribution
Maintenance • DM Provides Continual Maintenance and Upgrades of System Components • Develop Publications and Documentation • User Manuals • Formal Component Documentation • Templates, Guidelines, and Policies • Fact Sheets and Bulletins
Demonstration Greg Gunther
System Summary • Easy Access to Datasets • Generate Metadata Quickly and Easily • Find External Data with Over 1000 WWW Links • Simplify Data Processing Tasks • Organizes Projects with Workspace Templates • Streamlines CDROM Publications • Provides One-Stop Shopping For Shared Internal Resources
Future Directions • Increase Inventory Effort • Integrate GeoDatabase Model (ArcGIS) for Proprietary Datasets • Formalize Metadata Extension to FDGC Standard • Streamline Product Delivery - Implement IMS • Publish Documented Tools and Utilities • Implement Enterprise Architecture and Planning
Future Architecture Enterprise Planning* Getting Started Planning & Initiatives Current Systems BusinessProcesses Where We Are Today Data Architecture IS/IT Architecture GIS &Application Architecture Where We Want To Be Implementation and Migration Plans Plan To The Future *Modified from Spewak Model
Conclusions – What We Have Learned Data Management: • It’s ESSENTIAL for Survival But Needs to be Promoted • Distributed Projects REQUIRE Data Centralization • Projects RARELY Account for Data Management Planning and Costs • Data Stewardship MUST Begin at the Onset of Projects • The Terms EASY and USEFUL - Lead to Implementation • Component Model Must be FLEXIBLE to Adapt to Technology Trends
The End And The Beginning Of a New Cycle…