Evaluation: Exploring a Reporting Framework – The Community Nutrition Education Logic Model

1. Evaluation: Exploring aReporting Framework –The Community Nutrition Education Logic Model Helen Chipman Program Coordinator, FSNEP, CSREES/USDA Prepared for ASNNA Post-Conference, Washington DC 28 February 2003

2. Getting to the Same Page • Purposes – To answer questions • What are we getting for our nearly 400 million dollar investment (199 million food stamp administrative dollars)? • Are we making a difference with our target population? • Why Developed • Needed for CES/Land Grant System – a national report that reflects impact without loosing the richness of program diversity Presentation by Helen Chipman, February 2003

3. Getting to the Same Page • Framework – to report what happens in the short, medium and long term, with respect to individuals/households, institutions/communities, social structures/policies • Outcomes – core areas identified by FNS commissioned white papers • Theoretical basis, use of well known model Presentation by Helen Chipman, February 2003

4. Getting to the Same Page • Developed within Extension/Land Grant System; others had opportunity to provide input, recommendations incorporated into the model (all networks, youth evaluation workgroup, FNS/Food Stamp Program - nationally) • FNS has accepted use of the CNE Logic Model, with disclaimer – not exclusively Food Stamp Nutrition Education Logic Model Pesentation by Helen Chipman, February 2003

5. Getting to the Same Page • Does not replace reports/information requested by FNS • Information requested should be readily available for transfer to other reports, whoever the stakeholder is • Pilot effort – learn what states have; refine based on what is already in place • Identify gaps (in states and nationally) - strengthen planning/implementation Presentation by Helen Chipman, February 2003

6. Getting to the Same Page • Is something where we had nothing! • Is live – the www.csrees-fsnep.org website contains: • Two training modules: 1) logic models in general; 2) CNE Logic Model specifically • The CNE Logic Model (module 2, section 3) • Worksheet for states to use in preparing state reports (module 2, section 3) Presentation by Helen Chipman, February 2003

7. The State Report • For Extension/Land Grant System; others can use if desired • Will be used to communicate regionally and nationally about Food Stamp Nutrition Education – to tell the story • Information from FY 2002 – make it strong and complete, but brief • States can send 1 or 2 reports (may or may not send separate network report) Presentation by Helen Chipman, February 2003

8. The State Report • State coordinators/directors or their designees to complete • Email attachment to secretary of Helen Chipman • Contractor will aggregate into regional and national reports; also provide feedback to states Presentation by Helen Chipman, February 2003

9. The State Report • Projected timeline* • April 15, 2003 – report due to Helen’s secretary • June 30, 2003 – feedback to states • August 15, 2003 – regional and national reports available on the national website • Actual times depend on what we receive and contractual arrangements we are able to make to develop the reports Presentation by Helen Chipman, February 2003

10. State Reports – Network Considerations • Are considerable number contracted through the Extension/Land Grant System – important to report • Socio-ecological model partially collapsed to reduce complexity (institutions/communities) • Endpoints – model doesn’t resolve the ongoing discussion; determined from FNS guidance/research Presentation by Helen Chipman, February 2003

11. Steps Taken to Explain “How to Complete the Worksheet” • Asked Joyce Counihan to review the worksheet; comments incorporated into web-based document • Reviewed “Food Stamp Nutrition Education Networks – Partners for Better Health” Publication • Completed the worksheet based on information state Networks provided, national statistics, and some extrapolation Presentation by Helen Chipman, February 2003

12. Situation Statement: Description of State’s Food Stamp Population • State population – 6 million people • State poverty rate – 15% • State food stamp participation 60% of 1,000,000 eligible; average of $185 food stamp benefits/household/month • Children & seniors disproportionately represented ____ % children; ____ % seniors; ____ single parent homes… • Gender, race, household composition… Sample ideas shared by Helen Chipman, February 2003

13. Situation Statement: Description of State’s Food Stamp Population, cont. • Dietary Quality: Substantial percentages of households falling short of RDAs – 69% of iron, 79% of folate • Food Security – In 2001, 50% of food stamp participants experienced some level of food insecurity, which they attributed to lack of financial resources (ref); 30 thousand experienced outright hunger (ref) Sample ideas shared by Helen Chipman, February 2003

14. Situation Statement: Description of State’s Food Stamp Population, cont. • Rising demands at food banks – emergency food providers around the state attribute the increase in demand to working families; the faltering economy, and… Sample ideas shared by Helen Chipman, February 2003

15. Situation Statement: Description of Issues of Concern • Dietary Quality • Poor iron absorption despite high iron intakes – question link to enriched and fortified foods • Increased obesity rates • Decreased meals together as families • Low intake fruits and vegetables • General poor nutrition among seniors Sample ideas shared by Helen Chipman, February 2003

16. Situation Statement: Description of Issues of Concern, cont. • Food Security • Lack of food access • Transportation issues • Other barriers (list) • Food Safety • Identify • Shopping Behavior/Food Resource Management • Identify Sample ideas shared by Helen Chipman, February 2003

17. Situation Statement: State’s Objectives for FY 2002 • Encourage increased fruits and vegetable consumption by limited resource individuals • Increase consumption of low-fat, calcium-rich foods • Increase physical activity of limited resource individuals by 3% • Connect state objectives to Healthy People 2010 Sample ideas shared by Helen Chipman, February 2003

18. Situation Statement: State’s Objectives for FY 2002 • Increase participation in Food Stamp Nutrition Education • Increase participation in the Food Stamp Program by eligible persons • Strengthen the abilities of community nutrition coalitions to increase healthful nutrition and physical activity practices among low-income audiences Sample ideas shared by Helen Chipman, February 2003

19. Inputs: Financial Resources – Budgeted Dollars • FNS Funds: $499,000 • Matching Funds, state: $260,000 • Matching Funds, local/other: $241,000 • Total: $1,000,000 Sample ideas shared by Helen Chipman, February 2003

20. Inputs: Financial Resources – Budgeted Dollars, comments • State funds – State government and university, state grants • Local funds – County government, local network partner funds, local grants • Does not include private in-kind contributions, other federal funds • Additional state/local contribution of $50,000 not included in cost share Sample ideas shared by Helen Chipman, February 2003

21. Inputs: Planning Process • Conduct needs assessment/analyze the situation • Segment and select the target audience • Conduct formative research • Set goals and objectives • Develop marketing and communication strategies • Conduct concept and content pre-testing Sample ideas shared by Helen Chipman, February 2003

22. Inputs: Planning Process, cont. • Develop promotion plan and communication materials • Pre-test, refine, and produce materials • Implement program/conduct social marketing campaign • Conduct process and outcome/impact evaluation at all stages • Revise as needed Sample ideas shared by Helen Chipman, February 2003

23. Inputs: Materials – Curriculum Sample ideas shared by Helen Chipman, February 2003

24. Inputs: Materials – Educational Media • Include content that is not as extensive as a specific curriculum and not part of social marketing campaign Sample ideas shared by Helen Chipman, February 2003

25. Inputs: Materials – Social Marketing Campaign • Jump ‘N Jive... Come Alive with Fruit -- includes tip sheets, interactive displays, posters, newsletter articles, school lunch menu backs, food demonstrations and food sampling, stickers and balloons… Contact (name, institution/agency, state, email address, phone) Sample ideas shared by Helen Chipman, February 2003

26. Inputs: Materials – Social Marketing Campaign • Pick a Better Snack™ -- promotes fruits and vegetables as snacks; includes public service announcements recipes, newspaper/newsletter articles, videos… Contact (name, institution/agency, state, email address, phone) Sample ideas shared by Helen Chipman, February 2003

27. Inputs: People – Expertise • Steering committee – professionals with expertise in nutrition, management, marketing, finances, education…. • Network composition – number of persons, description of expertise, etc. Sample ideas shared by Helen Chipman, February 2003

28. Inputs: People – Accountability • Routine fiscal, personnel, affirmative action audits within university/ institutional system; quarterly reports to grants and contracts office; time and effort reporting (plan confirmation/ other) • Periodic reporting to/from partner agencies (describe) Sample ideas shared by Helen Chipman, February 2003

29. Inputs: People – Accountability, cont. • Accountability procedures within partner infrastructures (describe) • Annual face-to-face meeting with state food stamp fiscal office representative Sample ideas shared by Helen Chipman, February 2003

30. Inputs: People – Intra-Institutional Relationships (STATE Level) • Collaboration between your state Food Stamp agency and other state agencies that administer FNS and other USDA programs • Team Nutrition/Food Stamp Program Partnership – Why needed, how compliment, but do not duplicate • WIC – FSNEP Partnership – Why needed, how compliment, but not duplicate • EFNEP – Same • Others Sample ideas shared by Helen Chipman, February 2003

31. Inputs: People – Intra-Institutional Relationships (STATE Level), cont. • Head Start Consortium – access to low-income parents of young children • Fruit & Vegetable Growers Association – provide foods for social marketing campaign to low income families • 5 African American Organizations, 3 Tribal Organizations, 2 Faith-Based Organizations – access to limited resource population Sample ideas shared by Helen Chipman, February 2003

32. Outputs: Individual/Household Activities Sample ideas shared by Helen Chipman, February 2003

33. Outputs: Individual/Household Participation Sample ideas shared by Helen Chipman, February 2003

34. Outputs: Individual/Household Participation – Comments • Direct methods – 12 lessons per child – 3-5 years old – in head start classes • Indirect methods – posters, bookmarks, signage, billboards newsletters, calendars, radio, and newspapers Sample ideas shared by Helen Chipman, February 2003

35. Outputs: Institution/Community Activities Sample ideas shared by Helen Chipman, February 2003

36. Outputs: Institution/Community Participation Sample ideas shared by Helen Chipman, February 2003

37. Outputs: Institution/Community – Comments • School districts – primarily middle school and high schools • Private – supermarkets, neighborhood markets, and warehouse stores Sample ideas shared by Helen Chipman, February 2003

38. Outputs: Social Structures, Policies, Practices – Activities Sample ideas shared by Helen Chipman, February 2003

39. Outputs: Social Structures, Policies, Practices – Participation • Many Faces of Food and Agriculture – bus tour with continuing dialogue: included network stakeholders, industry leaders, and public officials to provide first-hand look at food production, delivery, and consumption issues in low-income area of state, with special attention to examining impact of issues and the system on low income citizens Sample ideas shared by Helen Chipman, February 2003

40. Impact Statement -- # 1 • Objective – 3-5 year old children try new foods • Indicator ID – DQ 1 • Core Element – Dietary Quality • Level of Intervention – Individual/Household and/or Community/Institutions • Time Frame – Short Term Sample ideas shared by Helen Chipman, February 2003

41. Impact Statement #1, cont. • In 2002, preschool children who attended head start or preschool participated in a 12-week “new-foods” intervention class. Of the 432 children that participated in a “tasting party” evaluation at the conclusion of the class, 86% tried all four of the novel foods; 96% tried two of the novel foods. Sample ideas shared by Helen Chipman, February 2003

42. Impact Statement #1, cont. • What is the significance of this impact, e.g. why is it important that children try new foods? • How representative is this number of preschool age children in the state – geographic area, pilot project, convenience sample? • Does the number represent all children who participated in 12-week intervention? Sample ideas shared by Helen Chipman, February 2003

43. Impact Statement #1, cont. • How many children shared the information with their parents later? • What was the impact for parents – did they request additional information, did they offer the new foods at home? • Did the new foods become part of the institutions’ menu options? • Any evidence that the foods are being eaten repeatedly after exposure? Sample ideas shared by Helen Chipman, February 2003

44. Impact Statement #1, cont. Data Collection Method • Teachers tracked children’s responses at conclusion of 12-week intervention • likes it • doesn’t like it • didn’t try it • At end of class or after class had been completed? Sample ideas shared by Helen Chipman, February 2003

45. Impact Statement #1, cont. Tools • Survey • Developed through formative research? • How validated? Sample ideas shared by Helen Chipman, February 2003

46. Impact Statement -- # 2 • Objective – Individuals eat more healthy by eating breakfast • Indicator ID – DQ 1 Could also be DQ 18 or 19 • Core Element – Dietary Quality • Intervention – Individual/Household Could also be Community/Institution • Time Frame – Medium Term Sample ideas shared by Helen Chipman, February 2003

47. Impact Statement #2, cont. • Between 2000 and 2002, 74% of participants became aware of the campaign and 12% reported increasing their frequency of eating breakfast as a result of the campaign. Sample ideas shared by Helen Chipman, February 2003

48. Impact Statement #2, cont. • Why is eating breakfast important? Tie back to dietary quality. • Who was targeted – general public, low income public, others? Clarify. • How do you know you reached your target audience? • How frequently were they eating breakfast? Any baseline comparisons of how much change this represented? Sample ideas shared by Helen Chipman, February 2003

49. Impact Statement #2, cont. • What about community organizing impact – numbers and types of partners who joined to promote eating breakfast – those who were part of the social marketing campaign? • What did the 300+ public and private partners do differently because of Food Stamp Nutrition Education? • Any lasting commitment or further action? Sample ideas shared by Helen Chipman, February 2003

50. Impact Statement #2, cont. Data Collection Method • Secondary data • Telephone contacts • In-depth interviews • How wide scale was the campaign – the whole state; a targeted area, etc.? • Describe more specifically how you arrived at the % -- random sampling, convenience, etc. Sample ideas shared by Helen Chipman, February 2003