Using the ODSS to Report Progress and Impact Iowa SPF-SIG Initiative May 30, 2013

1. Using the ODSS to Report Progress and Impact Iowa SPF-SIG Initiative May 30, 2013

2. Part I: Reporting Your Results • Part 2: Using ODSS Data to Answer Evaluation Questions • Part 3: Using the ODSS to Create Graphs

3. Part IReporting Your Results

4. What is providing a report? • Presenting data from the evaluation to those involved in the initiative • Usually presented in graph form • Should be done on an ongoing basis; can also be done at the end of an evaluation

5. Why provide a report? • To assess progress towards goals • To see where more energy may be needed and where too much energy is being spent • To focus on and celebrate cumulative accomplishments • To show funders your accomplishments and provide them the opportunity to help re-direct the initiative

6. When should you report out? • Your staff/volunteers should get feedback at regular intervals, especially early in the initiative's development. • You might want to have feedback sessions on a quarterly (four times a year) basis.

7. Present the information • Introduce the information (definition, example, and why it's important) • Give a detailed example from recent accomplishments • Point out what's positive • Make note of any trends • Discuss re-direction of efforts • Review examples on graph • Note evidence of progress • Ask for comments or questions from the group • If the measure doesn't show recent growth: Show any previous growth

8. Overall tips • Present data as a good thing • Focus on the positive • Convey the following: • optimism about prospects for success (if appropriate) • the need for changes or adjustment (if appropriate) • a shared vision of the initiative as an effective catalyst for change

9. Reasons for informing the public at the local level: • Raise awareness • Attract resources • Promote registration • Lobby for ordinances or program changes • Provide accountability

10. Three tips for making sure your findings aren't ignored: • Give your information to the right people! • Address issues which those people think are important • Be sure the information is presented in time to be useful and in a way that's clearly understood

11. What are some key audiences for the data? • Staff • Volunteers • Supporters in the community • Funders • Your target population • The general public

12. Part 2 Using ODSS data to answer evaluation Questions

13. Challenges • A key challenge in evaluating comprehensive community health initiatives is how to document and illustrate the unfolding of the intervention; that is, the multiple changes in the environment that can lead to widespread behavior change and improvement in population-level outcomes.

14. Illustrative evaluation questions • What actions have we taken to bring about new/modified programs/policies/practices? • Is the initiative a catalyst for change? • How are our efforts contributing to desired outcomes? • Which intervening variables are most important in our efforts? • How are we bringing about changes among community sectors? • Are we seeing changes in underage drinking? • What policies were changed or new policies enacted to restrict alcohol use?

15. Linking graphs to evaluation questions • What new/modified programs and practice changes occurred in the community?

16. How are we contributing to changes in our outcomes?

17. Illustrative examples of programs and practices • In collaboration with SPF SIG and the police department, the County passed laws regulating happy hours and sales of alcohol • For the period of August 7-August 14, 2012, the County SPF SIG ran the underage drinking radio ad for the first time on KUOO 5 times/day for 7 days consecutively.

18. How are we bringing about changes among community sectors?

19. How are our efforts contributing to desired outcomes?

20. On which intervening variables are we focusing our efforts?

21. On which strategies are we focusing our efforts?

22. Part 3 Using the ODSS to Create Graphs

23. Creating Graphs • Using online graphs, community and evaluation partners can examine the rate of community/system changes over time, or allow for systematic reflection on the proportion of changes occurring in different sectors.

24. Creating Graphs • To begin, click on the CREATE Graph button on your main ODSS page.

25. Creating New Graphs • The next page will display:

26. Creating New Graphs • Create a new graph: • Used when creating a graph for the first time or when creating a graph with a unique set of data. • Graph from template: • Used when creating a graph that will use the same settings as a previous graph. Once a graph is created, the option exists to save it as a template to be used for future graphing. (For example, once you have created a graph of Accomplishments, you can save it as a template, and use this template for any future graphs of Accomplishments.)

27. Creating New Graphs • Use the drop-down menu to identify the data you want to graph. • Note: This drop-down menu will feature any ODSS data or list data available on the site.

28. Example of Community Change Cumulative Line Graph • When creating a CC cumulative line graph, you would select the following options from the drop-downs and then Create Graph:

29. Example of Community Change Cumulative Line Graph • You will then be presented with a graph for which you can change the date range, date display, labels display, and graph type

30. Date Range: • Adjust the range of data displayed by changing the dates shown. You can type the dates in, or use the calendar that appears when you click within the box. • Date Display: • For a broader or narrower view of the data points throughout the designated date range, you can change the Date Display to reflect data points by Day, Month, Quarter or Year. • Labels Display: • Display or hide data labels on the graph to help identify the values shown in each data point of the data series. If using a pie graph, the option will exist to Show Percentage.

31. Types of Graphs • If you would like to view varying presentations of the data, you can modify the graph type by selecting from the Graph Type icons. • Line Graph: • Line graphs are used to track changes over periods of time or cumulative changes. • Pie Chart: • Pie charts are best to use when you are trying to compare parts of a whole or show a distribution of categories. • Bar Graph: • Bar graphs are used to compare data between different groups or to track changes over time. • When trying to measure change over time, bar graphs are best when the changes are larger.

32. Edit Graph • When you click the Edit Graph button, this will open a new window with options to Edit Basic Information, Edit Data, Add Additional Data, Add Additional Sites, and Add Comparison Graph. Simply rollover the category you want to edit, and then type in the new information.

33. Edit Basic Information • Title • Update the title of the graph. • Note: The default title will always be the name of the data selected for the graph (e.g., Accomplishments, Participant Data).

34. Date Range • Adjust the range of data displayed by changing the dates shown.

35. Date Display • To review a broader or narrower view of the data points throughout the designated date range, you can change the Date Display to reflect data points by Day, Month, Quarter or Year.

36. X and Y Axis Label • Update the label for the X Axis and Y Axis.

37. Labels Display • Click here to either display the data labels on the graph, or keep them hidden.

38. Graph Type: • If you would like to view varying presentations of the data, you can modify the graph type by selecting from the Graph Type icons.

39. Edit Data

40. Legend • This field allows you to modify the legend of the graph. • Note: The default for the legend will automatically be the name of the data selected for the graph (e.g., Accomplishments, Indicators, etc.)

41. Data Source • This is the source of your data (e.g., Accomplishments, Indicators, etc.)

42. Observer • Observer 1: Primary Data • Observer 2: Reliability Scorer Data • Observer 3: Calibration Scorer Data

43. What specific information do you want to graph? • This field is where your data is coming from, i.e., what you are graphing (Code, in this example). You can also filter by Code when selecting the specific information to graph

44. Add Additional Data • Select from the data sources on your site to include additional data in your graph.

45. Add Additional Sites • Select additional sites from within the site collection of your WorkStation to layer data in your graph. • Please note that you are able to select multiple sites (e.g., different programs or locations) and series (e.g., different data sets) from which to graph.

46. You also have the option to add a comparison graph. For instance, if you want to compare Accomplishments from this year to the previous year, you can choose last year’s graph from the Select Comparison Graph drop-down menu and hit Save. This will return you to the graph display, and you will see the graph that you just created on the left side, and the comparison graph that you just added on the right. • Once you have added a comparison graph, you can also edit this graph, by selecting Edit Comparison Graph. If you decide you no longer want the comparison graph, you can simply click the Edit Graph button again, and under Add Comparison Graph, you can choose No graph under the Select Comparison Graph drop down menu. • Once a comparison graph is selected, you will be able to view the two graphs stacked within the window that appears.

47. During the graph creation process, you may find that you periodically create graphs with the same set of criteria. To help save time during this process, you can create a template from a graph. You can select Manage Templates from either the web part menu or Graph Management:

48. The Create Template button will allow you to create your new template. This page will also show all existing templates that you can choose from to create a new graph. • Once you have created a template, the next time you create a graph, when prompted Create a new graph, or Create graph from template, you can select the template created from the drop-down menu.

49. To view all the graphs that have been created on the site, click the VIEWGraphs button on your main ODSS page.

50. Viewing Graphs • This window will display all available graphs on the site.