Design and Evaluation of Coastal Web Atlases

1. Design and Evaluation of Coastal Web Atlases Steve Ventura Land Information and Computer Graphics Facility University of Wisconsin-Madison David Hart University of Wisconsin Sea Grant Institute ICAN-Great Lakes 2010 Workshop September 13-15, 2010 Madison WI

2. Objective: Benefits of a structured process to design and evaluate a coastal web atlas • Project implementation and evaluation approaches • LOGIC models • Wisconsin history and context – steps to • Wisconsin Coastal Atlas and LOGIC model • Discussion: project evaluation

3. Implementation and Evaluation Approaches • Structured systems analysis and design method • UK government, early 1980s, large IT projects • Stages • Feasiblity - technical, financial, organizational, ethical • Evaluation of the current environment - operations, data, technology • System options - automation, centrality, transition process, costs/benefits • Requirements specifications - process specifications, user needs, data models, entity/relation diagrams • Technical options - hardware, software, staff, space, user interface • Logical design - data catalog, data structure • Physical design - translating to implementation and specifications

4. Implementation and Evaluation Approaches • SSA adapted to GIS • Numerous examples • e.g. Ventura, 1992

5. Implementation and Evaluation Approaches

6. Software and user interface development • Standard process: requirements, specification, architecture, design, implementation, testing, deployment, maintenance • Many approaches • Waterfall - structured process: requirements, design, development testing, installation, maintenance • RAD - rapid application development: structured process with iteration and rapid prototyping • Spiral - structured process, rapid prototyping, risk analyses • Agile - customer satisfaction orientation, focus on interface evaluation

7. Process Improvement Models • CMMI - Capability Maturity Model-Integrated • guide and improve processes such as system development and deployment • ISO 15504 - Software Process Improvement Capability Determination (SPICE) • framework for the assessment of software processes

8. Process Improvement Models • CMMI - Capability Maturity Model-Integrated • guide and improve processes such as system development and deployment

9. Process Improvement Models • ISO 15504 - Software Process Improvement Capability Determination (SPICE) • framework for the assessment of software processes • Organizational • Management • Engineering • Acquisition & supply • Support • Operations

10. Logic Models • Project and program planning, development and evaluation • Tool developed by University of Wisconsin Extension • Widely adopted and adapted - including coastal management programs

11. Logic models to enhance program performance Ellen Taylor-Powell, Ph.D. Evaluation Specialist University of Wisconsin- Extension-Cooperative Extension

12. Logic model is a… Picture of your program or intervention Graphic representation of the “theory of action” – what is invested, what is done, and what results Core of planning and evaluation Provides a common framework for your work

14. Wisconsin History and Context • Wisconsin Land Information Program • 1989 state law to modernize land information • Research • GIS implementation - Niemann, Ventura, Kuhlman, Tulloch… • Data integration - Hart… • Coastal issues - Edil, Mickelson, Scarpace… • Sea Grant coastal outreach program

15. WI Sea Grant coastal outreach program • Enhancing local GIS capacity • GIS for planning and management • Linkage to resource management tools • Hands-on training • Comprehensive, dynamic GIS For coastal management • Web-based tools and data for coastal decision-making • Visualization of coastal processes • GIS-based land use planning in coastal communities • Citizen access to data and planning tools • Web-mapping services • Multi-jurisdictional data integration • Wisconsin Coastal Atlas

16. Sea Grant coastal outreach program

17. Wisconsin Coastal Atlas - Geospatial Data Portal • Continuing to build “dynamic, distributed” GIS • Web-portal interface and data access tools • demonstrate solutions to institutional and technical barriers • data cataloguing and archiving processes • ontology tools to promote semantic integration • Evaluation and comparison of interface tools • Web mapping engines • Interface development tools • Cartographic design principles and examples • Replicable structured process for design and evaluation of CWAs (logic model template)

18. Wisconsin Coastal Atlas - LOGIC MODELING • (subset of) Project Design and Evaluation course • NOAA Coastal Services Center - Pam Kylstra, Trainer • June 7-8, 2010 in Madison • Instructional Design Theory • ADDIE Model • Assessment and Analysis • Design • Development • Implementation • Evaluation • (and iterate)

19. Wisconsin Coastal Atlas - LOGIC MODELING • (subset of) Project Design and Evaluation course • NOAA Coastal Services Center - Pam Kylstra, Trainer • June 7-8, 2010 in Madison • In reality: well-facilitated brain-storming leading to a logic model • Diagrammatic representation of project objectives and activities • Common understanding between multiple participants • Still need to flesh out evaluation components

21. Evaluation • Logic model provides framework • Choose appropriate type(s) of evaluation • Evaluate at the correct level • Evaluate to determine merit of activity, worth of outcome • Make decisions about the project based on evaluation results • Increasingly difficult to assess as you move from outputs to short, medium, long-term outputs

24. Evaluation • Tools and Instruments • Interviews and focus groups • Questionnaires and surveys • Checklists, product review • Use observations • User tests, concept maps, rubrics • Peer review • Expectations • Use • Change in knowledge, attitudes, skills • Change in intent to behave • Change in behaviors or activities

25. Evaluation • WHAT WORKS FOR COASTAL ATLASES? • YOUR THOUGHTS

26. THANK YOU • And thanks to • UW Sea Grant Institute • NOAA Coastal Services Center • University of Wisconsin Extension