NCGDAP: Building Processes for Geospatial Data Archiving in North Carolina

1. Collection Building Processes within the North Carolina Geospatial Data Archiving Project (NCGDAP)NCSU LibrariesSteve Morris Head of Digital Library Initiatives Digital Preservation in State Government: Best Practices Exchange 2006

2. Overview • Project context and targeted content • Engaging spatial data infrastructure • Data exchange methods • Rights issues Note: Percentages based on the actual number of respondents to each question

3. NC Geospatial Data Archiving Project • Partnership between university library (NCSU) and state agency (NCCGIA), with Library of Congress under the National Digital Information Infrastructure and Preservation Program (NDIIPP) • One of 8 initial NDIIPP partnerships (only state project) • Focus on state and local geospatial content in North Carolina (statedemonstration) • Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventories • Objective: engage existing state/federal geospatial data infrastructures in preservation Note: Percentages based on the actual number of respondents to each question

4. Scale of the Problem • County Digital Orthophotos • 88 counties with estimated 154 flights by 2006 • Estimated 30 gb/flight – 4.6 TB total • County, City, COG Vector Data • Variable mix of layers; some continuous update • 92 of 100 counties with GIS systems • 51 municipalities with GIS systems • State Agency Data • 1993 and 1998 statewide orthos – 800 gb • Terabytes of vector data and other imagery • 17-20 TB of LIDAR data Note: Percentages based on the actual number of respondents to each question

5. Earlier NCSU Acquisition Efforts • NCSU University Extension project 2000-2001 • Target: County/city data in eastern NC • “Digital rescue” not “digital preservation” • Project learning outcomes • Confirmed concerns about long term access • Need for efficient inventory/acquisition • Wide range in rights/licensing • Need to work within statewide infrastructure • Acquired experience; unanticipated collaboration Note: Percentages based on the actual number of respondents to each question

6. What is Spatial Data Infrastructure? • National Spatial Data Infrastructure • Content Standards, metadata standards • Data discovery methods (Geospatial One-Stop, NSDI Clearinghouse Z39.50 search) • Cultivating development of framework data layers • State Spatial Data Infrastructures • State level metadata development, content standards, data sharing, data clearinghouses • NC OneMap in North Carolina, from 2003 SDI’s have not been addressing archiving and preservation … Note: Percentages based on the actual number of respondents to each question

7. SDI and Collection Building • Leverage data sharing agreements • Data content standards – adherence by agencies makes archiving easier • Archive feedback to metadata outreach efforts • Content exchange networks (not a well developed part of SDI) • Goals • Make data more preservable at point of production • Make preservation a seamless part of the data production process Note: Percentages based on the actual number of respondents to each question

8. Transfer modes - Conventional • CD/DVD • 230 CD-ROMs for 1999 Wake County orthophotos • External drives • Becoming more routine • FTP • Bandwidth intensive: restricted to off hours, or not done • WAN (Wide Area Network) • Network incompatibilities, network load • Web Download • Complex interfaces make automation difficult Note: Percentages based on the actual number of respondents to each question

9. Complex download interfaces make automated web capture difficult Note: Percentages based on the actual number of respondents to each question

10. Transfer Modes - Web Services • WMS (Web Map Service) • Can only capture derived static images, losing the underlying data intelligence • Possible use for agent-based image atlas creation • WFS (Web Feature Service) • Transfers actual vector data as GML • Not widely deployed; variation in configuration • Scalability for bulk transfer questionable • Federal Enterprise Architecture Geospatial Profile suggests WMS, WFS, FTP Note: Percentages based on the actual number of respondents to each question

11. Harnessing Geospatial Web Services Image atlases from WMS services? Capturing cartographic representation? Recording records from decisions-making processes? Later: data transfer via WFS & GML?, Other? Note: Percentages based on the actual number of respondents to each question

12. SDI Approach to Transfer Problem? • 20 different NC state agencies ask for local data, including at least three units from one agency • Data also transferred to federal agencies (Census, FEMA, …) for data improvement • … and to regional agencies (RPOs, MPOs, COGs) for data aggregation and projects • Archive development should piggyback on existing data transfers • Grass roots effort in NC to coordinate acquisition, formalized in working group, starting April 2006 Note: Percentages based on the actual number of respondents to each question

13. Intellectual Property Rights Issues • Subject to Public Records Law • Public record: no privacy issues … • … but records for some individuals may be filtered • Disclaimer viewing important (liability) • Restrictions on commercial reuse – desire for downstream control of data • Great deal of variation in access/use policy • Trust between agencies is important • Becoming more common to share data but not sign formal agreements Note: Percentages based on the actual number of respondents to each question

14. Obtaining NC Local GIS Data Source: NC OneMap Data Inventory 2004 Note: Percentages based on the actual number of respondents to each question

15. Rights issues in the web services space are ambiguous -- e.g., 39 NC counties allow GIS-based access to ArcIMS, but extraction rights are not clear Note: Percentages based on the actual number of respondents to each question

16. Conclusion • State and local geospatial data a very large scale problem (data quantity; number of agencies) • Need to engage spatial data infrastructure in collection progresses, piggyback on existing processes • Web services processes may play a role in future collection building efforts • Great deal of variation in interpretation of public records law, even in one state Note: Percentages based on the actual number of respondents to each question

17. Questions? Contact: Steve Morris Head, Digital Library Initiatives NCSU Libraries Steven_Morris@ncsu.edu Web site: http://www.lib.ncsu.edu/ncgdap/ Note: Percentages based on the actual number of respondents to each question