Introduction

1. Addressing Agency Database Needs: A Recent Experience From IllinoisDonald McKay, Damon Garner, Deborah Griest, Stephen Gustison, Donald Keefer, Robert Krumm, Alison Lecouris, Brent Lemke, Daniel Nelson and Barbara Stiff Illinois State Geological Survey

2. Introduction • Recent Effort to Access Data Management Needs of the Illinois St. Geological Survey • Internal Effort • Database Needs Assessment

3. Background Info. • Illinois Survey – about 200 people • 20 years as a GIS user site (Arc license #27) • More staff using technology – GIS, database, graphics, web to accomplish project goals • Oracle, Access, ArcGIS, Rockworks & more • Many concerns about database issues – access, archival of data, conversion from paper to digital, database structure, policies – both operational and administrative • Concerns about the agency’s ability to provide better support for IT operations – databases, GIS, web, more

4. Background Info. • Support staff stretched thin • Annual project planning cycle indicates key staff members are in very high demand • Oracle, ArcGIS, ArcIMS, SDE, application development, web • Many demands, expectations and desires • A recipe for disaster….? • To address concerns, a consultant to perform a review of database operations, assess needs, and make recommendations…? • Show me the (no) money • Decision was made to form an internal team to conduct the review

5. The Process • Form an internal team – Sept. 2003 • Database Management Coordinating Comm. • Chaired by Don McKay, Principal Geologist & Chief Scientist, an Experienced GIS User • Nine team members from many program areas • Scientists from Oil & Gas, Quaternary Geology, Groundwater, Environmental Studies and Assessments Sections • Key support staff from Geologic Records Unit and Financial Office • Technical staff - Oracle DB Admin, GIS DB Admin • Representation from many sectors – experiencewith database issues

6. Data Management Coordinating Comm. • Initial Meetings - Sept. 2003 • Identified issues including the need for easy desktop access to database holdings, consideration of data at all levels – digital and paper, level of staff support, training needs, application development needs, providing web access to data, and more

7. ISGS Mission Chief 3-D Geologic Mapping Carbon Management Centers Thrusts Data & Information Assets Products Applied & Basic Research Education,Tech Asst. & Outreach Operations & Support Sections Programs

8. Data Management Coordinating Comm. • Talk with staff – provide an opportunity to gatherinput from all staff • ‘Interviews’ with Managers and Program Directors of all Centers, Sections and Units – provide an opportunity to meet with staff members,listen to their comments and key concerns • Before the interviews, committee prepared a 3-pagelist of bullet points on various DB topics. The list of topics and questions served as a guidelinefor the interview sessions with Managers/Program Directors.

9. Discussion Topics -General • Imagine 5 years from now.  What would you like to be able to do with your projects and your data? • What are the principal barriers to capturing, automating or using the data needed or generated by your section? • What are your most important data mgt. needs?  • Which of your unit's important jobs would be most improved by better data management? • What examples of good data management in your section might be generalized for others in the Survey?

10. Discussion Topics – Data & Information • What do you consider to be data?  • What mission-critical data does your program require?  • What type of data does your program generate? • How much of the data that your program generates can be described as:- map/spatial data; tables of data; digital documents? - other digital images; hardcopy documents?- physical collections; other? • How much of your data use involves:- mapped views of your data?- tabular views or summaries of data?

11. Discussion Topics – Operations What input does your section do to add to your data?  • key enter data; scan/collect digital images; digitize vector maps; scan documents? • create reports; generate analyses; make interpretations? • import datasets; other, please specify. What are the ultimate outputs of your work flow?  • Images; maps; documents; reports; • analyses; interpretations • hard copy products; CD products • web-based presentations; web-based data delivery

12. Discussion Topics – Operations What are the essential analytical or processing functions and "objects" that occur within your data stream/work flow, between inputs & outputs? • Tabulation/reporting; statistical analysis; visualization; modeling; other? • Who in your section is involved in DB mgt. activities? If responsibility for coordinating import, organization, and management of your unit's data were centralized with one or more of your staff: • What barriers do you see to success? • What advantages do you see? What software are you using to: • Input; store; analyze; publish; archive • Software you would like to use for these functions?

13. Discussion Topics – Users and Customers Who are the known and potential users of your data in the following categories? • Your section/unit; Survey; External Please estimate the percentage of data access needs related to your programs that service: • Your section/unit; Survey; external  Of the consumers of your data/information: • Who are the most important? • Who are the most frequent? Please characterize your staff’s database management experience by number of individuals operating at: • beginning level; intermediate level; advanced level

14. Conducting the Needs Assessment • Database Team held 10 meetings (20 hrs.) • Met with 50 people • Received input from every program area within the agency – scientific staff & support staff • Received input from a broad cross-section ofpeople, in terms of DB/IT experience • Input from meetings, as well as written input – based on key issues (Data & Information,Operations, Users & Customers, Storage,Security)

15. Notable quotes • “I don’t know how to use existing tools or databases.” • “We are storing gigabytes of data on unstable media.” • “I can’t get help to do what needs to be done.” • “Our lab equipment is generating digital output, that is not being captured for lack of DB help.” • “Our data are misplaced, discarded, and orphaned, daily.” • “Our important applications are not maintained.” • “I can’t access important data, maps & records easily.” • “Huge amounts of our data on paper have no prospect of becoming digital.” • “Much of our data is completely undocumented.” • We heard it all – The Good, the Bad and the Ugly

16. Needs Assessment - Results • After meetings with staff, draft report was prepared • Presented to senior management team – March ’04 • Presented to all staff – May ’04 • Findings & Recommendations

17. Findings We have: • Huge data holdings • Tradition of digital data management • Needs outpacing support capabilities • Complex data management environment • Sophisticated & underserved users • Best available high-end software • Sufficient hardware & network

18. Findings cont. We Need: • A catalog of our data holdings • Easy access to our existing digital data • Help in automating our key data • Applications to allow us to interact with our data • A support infrastructure not understaffed and overwhelmed • Better overall DBM to • preserve our results • improve our projects & products • make our jobs easier • An effective information management policy • In short, we need “HELP!”

19. DBMCC’s Recommendations • Adopt a Bold Information Management Strategy • Assume New and Altered Roles and Responsibilities • Develop Needed Applications • Conduct More Frequent and Relevant Training • Increase DB Staffing and Change Hiring Practices • Increase Funding for Data Management • Maintain Adequate HW/SW Infrastructure

20. Adopt A Radically New Strategy • Focus on ISGS: serve ourselves 1st • Clarify & distribute DBM responsibility • Formalize a Database Management Oversight Team • Designate Center/Section/Unit Liaisons • Formalize a Core Database Staff

21. Strategy (cont.) • Collaborate on priority solutions • Hire expertise in DB Coordination & DB Design • Provide guidelines, tools, and techniques • Provide training • Implement coordinated distributed database management, including • Individual Data • Center, Section, & Project Data • Shared Data Repository

22. Assume New & Altered Roles Managers • Support improved DBM for the long haul • Provide appropriate staff & funding Oversight Team(advisory to Chief & SMT) • Represent all Survey sectors • Recommend Survey priorities Core DB Staff (Oracle & GIS DBAs, new hires) • Provide DB design & coordination • Provide guidelines & techniques • Maintain shared data repository

23. New & Altered Roles cont. Liaisons • Communicate priority needs • Become expert in appropriate tools • Oversee section/project data • Help section & project staff Everyone • Assess the status of your data • Learn to use appropriate tools • Adopt standards as they emerge • Prioritize & communicate your needs

24. Hiring Hire Key Expertise • Database Coordinator-Programmer • Database Designer-Programmer • Applications development - Mapping, Energy, Web, Business • Technical Assistants Adjust Hiring General Practices • Favor qualified, database-capable candidates

25. A database liaison represents and provides support for a given unit, project or service. Together, liaisons and core database staff cooperatively build the ISGS information system. The database management oversight team provides a formal mechanism for communication, input, and participation, and recommends policies and priorities to management. DMOT Unit- and function-specific databases are related yet distinct. They are developed as needed and are integrated with core and other databases as appropriate. Distributed Information Management, Liaison Role, and Coordination Environment

26. Database Applications Development* Core DB Staff, Application Developers, Liaisons • Develop New Needed Applications • Acknowledge/Document Current Applications • Various IMS Applications • SDE Support and Expansion • Web Development Applications • Share Applications Across Survey * Refers to both the graphic look and feel of the software interface and to the software and data manipulation functions to be performed.

27. Training Topics • Data Storage & Long-term Preservation • Data Organization & Quality Control • Data Documentation • Software Training • Data Management Guidelines • Special Topics (Oracle, ArcGIS, IMS)

28. Training Opportunities • General Software Training • Spreadsheets (Excel) • Databases (Access, Oracle) • Other (IMS, SDE, OCR Software) • Specific In-House Training • Data Management Guidelines • ISGS Data Resource Website

29. Overall Funding Recommendation The funding strategy should be flexible enough to provide project managers the ability to meet data management needs without placing unnecessary obstacles or regulations in their path.

30. Database GIS Support Network E-Mail Application Development PC Support Plotter Operations Web Specialized Software Existing IT Categories

31. Summary • ISGS recently completed a database needs assessment • An open process; entire organization was represented • Team of 10 staff members, lead by senior staff member • Interviews with 50 staff members & written input provided the basis for report to senior management • People came to the table with many issues • A tremendous amount of interest and enthusiasm

32. Summary • Critical issues: • Lack of easy access to digital and hardcopy data • Lack of a comprehensive catalog of database holdings • Poorly understood or non-existent policy • Support infrastructure overwhelmed by the rate of • acquisition of new data • A need for key application development to support • database use • Current funding and staffing limitations in state gov’t • necessitate a solution that leverages existing resources

33. Summary Proposed restructuring of Info. Mgt. approach –(1) decentralize routine information management tasks, i.e. allow/require units to directly manage their own data (2) centralize the oversight for information management by providing comprehensive plans, policies and guidelines (3) link centralized & decentralized components with DB liaisons (experts in administrative units) who provide communication for cooperative development, and who act collectively to execute agency policyNew hires were identified (DB Coord. & DB Designer/Programmer) Also, exploring ways to alter job descriptions and project funding to reflect a greater emphasis on DB/IT skills

34. Will it Work? Stay tuned for future developments Thanks to the Oregon Department of Geology & Mineral Resources