Assessing Need, Theory of Change, and Logic Model

1. Assessing Need, Theory of Change, and Logic Model 2014 AmeriCorps External Reviewer Training

2. Topics • Purpose of Logic Model • Assessing Need, Theory of Change and Logic Model

3. Preview of Assessment Questions (True/False) • The Logic Model is a visual representation of the applicant’s Theory of Change. • External Reviewers should assess whether the outcomes in the Logic Model are measurable. (Multiple Choice) • Data about the number of economically disadvantaged beneficiaries in a community would most likely describe: (Practice Your Understanding) • Complete portion of Review Assessment Form

4. Logic Model What is a Logic Model? • A visual summary or snapshot of your program that communicates how your program works, the resources you have to operate your program, the activities you plan, and the outcomes you hope to achieve • A well-specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice and describes the relationships among the key components and outcomes, theoretically and operationally (Notice of Funding Opportunity (NOFO) p. 18) • The purpose of a Logic Model is to describe how a program will create change.

5. Logic Model How a Logic Model Works Figure 1: How to Read a Logic Model • If these benefits to participants are achieved, then certain changes in organizations, communities, or systems might be expected to occur • If you accomplish your planned activities, then you will hopefully deliver the amount of product and/or service that you intended • If you accomplish your planned activities to the extent you intended, then your participants will benefit in certain ways • If you have access to them, then you can use them to accomplish your planned activities • Certain resources are needed to operate your program j k l m n Your Planned Work Your Intended Results

6. Logic Model Applicant Logic Models • Inputs – Resources, including AmeriCorps members • Activities (Interventions) • Outputs (Beneficiaries, other products of program services) • Outcomes • Short Term (Knowledge, Skills, Attitudes, Opinions) • Medium Term (Behavior) • Long-Term (Condition)

7. Logic Model Key Points • The Logic Model and narrative work together to depict the program’s Theory of Change • All the information in one doesn’t have to be in the other, as long as the two are aligned • The Logic Model is simply a visual representation of the Theory of Change and program design • Applicants do not measure all outcomes in Theory of Change; they continue to measure outcomes that make sense for their program; External reviewers do not assess applicants’ performance measures and so should not be concerned with whether outcomes are measurable • CNCS provided a template for applicants’ Logic Models

8. Assessment: Theory of Change Theory of Change Elements Community Problem/Need Specific Intervention IntendedOutcome • Evidence • Guides choice of intervention • Supports cause-effect relationship Statistics documenting the need

9. Assessment How Do The NOFO Pieces Fit? Applicant’s Overall Theory of Change • Need narrative • Theory of change narrative • Logic model

10. Assessment The Selection Criteria & Standards Four Key Themes • Completeness (Logic Model) • Alignment/Logic • Quality of Theory of Change • Level/Quality of Evidence

11. Assessment Review Rubric • Most items scored on a five-point scale • Some items scored on a two-point scale (Satisfactory/Unsatisfactory) • Refer to the Review Rubric in the handbook to read the descriptions for each Rating

12. Assessment Completeness (Logic Model) • Number of locations and sites where members are providing services • Number of members that will be delivering the intervention • Core activities that define the intervention or program model • Duration of the intervention • Dosage of the intervention • Target population for the intervention

13. Assessment How Assessed • Series of check boxes on the review form • Reviewers check the box if the item is present in the Logic Model

14. Assessment: Theory of Change Alignment with Logic Model • The Theory of Change is described consistently in the Logic Model and application narrative • The inputs, activities and outputs are logically aligned • The Theory of Change and Logic Model cover comprehensively the applicant’s entire program

15. Assessment: Theory of Change Quality of Theory of Change Problem • The community problem/need is prevalent and severe in the communities where members serve • The applicant cites specific, relevant data to document the need Intervention • The intervention is an effective way to address the problem/need identified by the applicant Outcomes • The outcomes identified in the Logic Model are logically aligned with the problem/need and intervention

16. Assessment: Theory of Change Assessing Alignment and Quality Alignment and Quality are assessed using a five-point scale: • Greatly exceeds the standard • Exceeds the standard • Meets the standard • Partially meets the standard • Does not meet the standard at all Refer to the Review Rubric to read the descriptions for each Rating.

17. Assessment: Need Need vs. Evidence Applicants are asked to provide data to demonstrate: • Prevalence and severity of need versus the level and quality of evidence • Specific, relevant data to document the need • Evidence to support the choice of intervention

18. Assessment: Need Community Problem/Need Data documenting problem/need should demonstrate the prevalence and severity of the problem: Who and how many are directly affected? How severe is the problem?

19. Assessment: Need Examples • Unemployment rate • Poverty rate • Percent of students eligible for free/reduced lunch • Population or size of area affected by natural disaster • Percent of adult population with diabetes

20. Assessment: Need Data Should Be • From a reputable source such as a government agency, foundation, university or research organization • Recent • Local

21. Assessment: Need Common Issues • State or national data is provided instead of local data describing the need in the specific communities where the service will occur • Statistics are unrelated to the problem/need • Statistics cited are not recent • Data sources are not identified or are not reputable

22. Next Steps To check for understanding and verify that you have completed this orientation session, please reference the remaining slides for the two-part Assessmentof this Orientation.

23. Assessment: Practice Practice Your Understanding • Answer the Assessment Questions at the following link: https://www.surveymonkey.com/s/logicmodels_assessment • Read and assess the norming/sample application and complete the Rating sections for the Problem/Need, Theory of Change, and Logic Model sections of the review form. Do not include the significant strengths, weaknesses, or other narrative sections. • You will receive a review form completed by a CNCS staff member. Compare your assessments to those in the key. • This will complete your verification requirement for this training module.

24. Assessment: Practice Practice SELF-ASSESSMENTWhen you receive the key for the practice activity from CNCS, compare your Ratings to those of the CNCS standard example. If your Rating was more than one Rating higher or lower than the example provided by CNCS, your assessment may not be normed closely enough to CNCS’ expectations for applicants