State Archives Meeting

1. North Carolina Geospatial Data Archiving Project/NDIIPP:Collection and preservation of at-risk digital geospatial data Partners:NCSU LibrariesProject Lead: Steve MorrisNC Center for Geographic Information & AnalysisProject Lead: Zsolt Nagy State Archives Meeting February 24, 2006

2. Project Context • Partnership between NCSU Libraries and NCCGIA under NDIIPP • Focus on state and local geospatial content in North Carolina (statedemonstration) • Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventory information • Objective: engage existing state/federal geospatial data infrastructures in preservation Note: Percentages based on the actual number of respondents to each question

3. Targeted Content • Resource Types • GIS “vector” (point/line/polygon) data • Digital orthophotography • Digital maps • Tabular data (e.g. assessment data) • Content Producers • Mostly state, local, regional agencies • Some university, not-for-profit, commercial • Selected local federal projects Note: Percentages based on the actual number of respondents to each question

4. Local Government GIS Landscape • Data resources are highly distributed and subject to frequent update • More detailed, current, accurate than federal/state data resources • North Carolina local agency GIS environment • 100 counties, 95 with GIS • 85 counties with high resolution orthophotography • Growing number of municipal systems • Value: hundreds of millions of dollars investment Note: Percentages based on the actual number of respondents to each question

5. Geospatial data types: Vector data Note: Percentages based on the actual number of respondents to each question

6. Geospatial data types: Aerial imagery Note: Percentages based on the actual number of respondents to each question

7. Geospatial data types: Aerial imagery Note: Percentages based on the actual number of respondents to each question

8. Geospatial data types: Aerial imagery Note: Percentages based on the actual number of respondents to each question

9. Geospatial data types: Tabular data (w/vector) Note: Percentages based on the actual number of respondents to each question

10. Time series – vector data Parcel Boundary Changes 2001-2004, North Raleigh, NC Note: Percentages based on the actual number of respondents to each question

11. Time series – Ortho imagery Vicinity of Raleigh-Durham International Airport 1993-2002 Note: Percentages based on the actual number of respondents to each question

12. Today’s geospatial data as tomorrow’s cultural heritage Note: Percentages based on the actual number of respondents to each question

13. Risks to Digital Geospatial Data • Producer focus on current data • Time-versioned content generally not archives • Future support of data formats in question • Vast range of data formats in use--complex • Shift to “streaming data” for access • Archives have been a by-product of providing access • Preservation metadata requirements • Descriptive, administrative, technical, DRM • Geodatabases • Complex functionality Note: Percentages based on the actual number of respondents to each question

14. Earlier NCSU Acquisition Efforts • NCSU University Extension project 2000-2001 • Target: County/city data in eastern NC • “Digital rescue” not “digital preservation” • Project learning outcomes • Confirmed concerns about long term access • Need for efficient inventory/acquisition • Wide range in rights/licensing • Need to work within statewide infrastructure • Acquired experience; unanticipated collaboration Note: Percentages based on the actual number of respondents to each question

15. Work plan in a Nutshell • Work from existing data inventories • NC OneMap Data Sharing Agreements as the “blanket”, individual agreements as the “quilt” • Partnership: work with existing geospatial data infrastructures (state and federal) • Technical approach • METS with FGDC, PREMIS?, GeoDRM? • Dspace now; re-ingest to different environment • Web services consumption for archival development Note: Percentages based on the actual number of respondents to each question

16. NCGDAP Philosophy of Engagement Provide feedback to producer organizations/ inform state geospatial infrastructure Take the data as in the manner In which it can be obtained “Wrangle” and archive data Note the ‘Project’ in ‘North Carolina Geospatial Data Archiving Project’– the process, the learning experience, and the engagement with geospatial data infrastructures are more important than the archive … What is the long term solution? Note: Percentages based on the actual number of respondents to each question

17. Expected Project Outcomes • Demonstration archive • Outreach activity – planting seeds • International, national, state, local, commercial • Learning experience, informing: • Spatial data infrastructure • Commercial vendors (data/software/consulting) • Repository software communities • Metadata practice (both GIS & preservation) • Rights management developments • Data and interoperability standards Note: Percentages based on the actual number of respondents to each question

18. Big Challenges • Format migration paths • Management of data versions over time • Preservation metadata • Harnessing geospatial web services • Preserving cartographic representation • Keeping content repository-agnostic • Preserving geodatabases • More … Note: Percentages based on the actual number of respondents to each question

19. Preserving Cartographic Representation Counterpart to the map is not just the dataset but also models, symbolization, classification, annotation, etc. Note: Percentages based on the actual number of respondents to each question

20. Project Status • Storage system and backup deployed • DSpace deployed • FGDC Metadata workflow finalized • Ingest workflow near finalization • Content migration workflow plan near finalization • Regional site visits planned for coming months • Wide range of outreach/collaboration: FGDC, ESRI, EDINA (JISC), USGS, OGC, TRB, etc. • Pilot project, georegistering digital archival geologic maps Note: Percentages based on the actual number of respondents to each question

21. Questions? Contact: Steve Morris Head, Digital Library Initiatives NCSU Libraries Steven_Morris@ncsu.edu Web site: http://www.lib.ncsu.edu/ncgdap/ Note: Percentages based on the actual number of respondents to each question

22. Risks to Digital Geospatial Data .shp .mif .gml .e00 .dwg .dgn .bsb .bil .sid Note: Percentages based on the actual number of respondents to each question

23. Metadata Availability Note: Percentages based on the actual number of respondents to each question

24. NC OneMap Initial Data Layers Produced by Cities and Counties Note: Percentages based on the actual number of respondents to each question

25. Content Identification and Selection • Work from NC OneMap Data Inventory • Combine with inventory information from various state agencies and from previous NCSU efforts • Develop methodology for selecting from among “early,” “middle,” and “late” stage products • Develop criteria for time series development • Investigate use of emerging Open Geospatial Consortium technologies in data identification Note: Percentages based on the actual number of respondents to each question

26. Content Acquisition • Work from NC OneMap Data Sharing Agreements as a starting point (the “blanket”) • Secure individual agreements (the “quilt”) • Investigate use of OGC technologies in capture • Explore use of METS as a metadata wrapper • Ingest FGDC metadata; Xwalk to MODS? PREMIS? • Maybe METS DRM short term; GeoDRM long term • Consider links to services; version management • Get the geospatial community to tackle the content packaging problem (maybe MPEG 21?) Note: Percentages based on the actual number of respondents to each question

27. Partnership Building • Work within context of the NC OneMap initiative • State, local, federal partnership • State expression of the National Map • Defined characteristic: “Historic and temporal data will be maintained and available” • Advisory Committee drawn from the NC Geographic Information Coordinating Council subcommittees • Seek external partners • National States Geographic Information Council • FGDC Historical Data Committee • … more Note: Percentages based on the actual number of respondents to each question

28. Content Retention and Transfer • Ingest into Dspace • Explore how geospatial content interacts with existing digital repository software environments • Investigate re-ingest into a second platform • Challenge: keep the collection repository-agnostic • Start to define format migration paths • Special problem: geodatabases • Purse long term solution • Roles of data producing agencies, state agencies; NC OneMap; NCSU Note: Percentages based on the actual number of respondents to each question

29. Rights Issues • Various interpretations of public records law • 53.9% of local NC agencies charge for data • 43.7% of local NC agencies restrict redistribution • Desire for downstream control of data • Disclaimer clickthrough; liability concerns • Filtered locations/individuals; post 9/11 issues • Restrictions on redistribution; commercial resale • Web services area in “Wild West” stage • Both content and technical agreements • GeoDRM initiative in the works Note: Percentages based on the actual number of respondents to each question

30. Big Challenges • Management of data versions over time • How to “get current object/metadata/DRM”? • Relation of the canonical metadata package to the ingest (and export) metadata package • Tailor to repository environment or make the acquaintance when needed? • Format migration paths (geodatabases, etc.) • Preserving Cartographic Representation • The counterpart to the map is not just the dataset—also models, symbology, interpretation, etc. Note: Percentages based on the actual number of respondents to each question

31. Geospatial data types: Satellite imagery Note: Percentages based on the actual number of respondents to each question