1 / 15

CaDEA Workshop 2 Input Brad cousins, university of ottawa

CaDEA Workshop 2 Input Brad cousins, university of ottawa. Welcome, introductions Conceptualizing the evaluation problem stakeholder interests Divergent and convergent processes Developing evaluation questions Program Logic Models Constructing the evaluation framework.

lyris
Download Presentation

CaDEA Workshop 2 Input Brad cousins, university of ottawa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CaDEA Workshop 2 InputBrad cousins, university of ottawa • Welcome, introductions • Conceptualizing the evaluation problem • stakeholder interests • Divergent and convergent processes • Developing evaluation questions • Program Logic Models • Constructing the evaluation framework

  2. Conceptualizing the Evaluation Problem • Program (Non-evaluator) Stakeholders • Individuals, groups or organizations with a stake in the program or its evaluation • Program structural and context considerations • Age, program theory, program organization openness to change, consensus among stakeholders, micro politics

  3. Conceptualizing the Evaluation Problem • Evaluator-Stakeholder relationships • External evaluation, internal evaluation and collaborative evaluation (Collab, Partic, Emp) • Relationships among non-evaluator stakeholders • Differential access to power, knowledge, expertise • Conflicting values perspectives • Resource considerations: fiscal resources, human resources, expertise (evaluation logic, program logic), time.

  4. Types of Stakeholders • Policy makers/sponsors • Program developers • Program administrators/managers • Program implementers • Intended program beneficiaries • Special interest groups • Others WHO ARE THEY? WHAT ARE THEIR INTERSTS? WHOSE INTERESTS COUNT?

  5. Divergent phase • Identify reasons for initiating the evaluation, issues, questions of interest • Identify contextual conditions: micro political analysis, evaluability assessment • List stakeholders and specify their interest and relative importance • Identify dimensions of performance (criteria) and standards • Describe object for evaluation • Tap multiple sources: • Stakeholders, models/frameworks, literature, professional standards, expert consultants, professional judgement.

  6. Divergent phase • Useful information from stakeholders • Perception of the program • Program purposes/goals • Program theory • Concerns • Evaluation questions • Intended uses of evaluation • Other stakeholders and their stake.

  7. Convergent strategies for planning and framing the evaluation • Identification of primary users and other stakeholders and their main interests, issues or questions. • Initial statement of the problem from primary stakeholders. • Other stakeholders take on the problem. • Generate list of rank ordered issues. • Attention to potential uses and decision rules. • Identify constraints and modify evaluation issues as appropriate.

  8. Formulate evaluation questions and issues • From convergent process, identify and determine priority of feasible list of questions • Implementation evaluation • Is the programme being implemented as intended? Why/not? • Process evaluation • Which aspects or components of the program are most potent in affecting desired outcomes? • Impact evaluation • Did the program meet its objectives? What unintended effects of the programme can be observed?

  9. Program Logic Model “A tool for describing program theory and for guiding program measurement, monitoring, and management” Rossi et al.

  10. Program Logic Models • Needs:Raison d’être for the program. The problem to be solved • Inputs: human, fiscal and other resources (e.g., partnerships, infrastructure) needed to run the program • Activities: all action steps needed to produce program outputs • Outputs: goods and services generated by the program (necessary but insufficient conditions required to realize program outcomes; can be counted) • Outcomes: link to program objectives; short-term or immediate, intermediate, long term (observed changes)

  11. Program logic model Outcomes Outputs Program Activities Inputs/resources Program Need

  12. ‘Results Chain’ Logic Model Area of ControlInternal to the Organization Area of InfluenceExternal to the Organization ImmediateOutcomes(Direct) IntermediateOutcomes(Indirect) Inputs(Resources) FINALOUTCOME Activities Outputs External Factors Efficiency Effectiveness

  13. Program Logic Models • Activities (service utilization plan): • Recruitment • Induction • Delivery • Regular program activities • Streaming • Supplemental activities • Terminal

  14. Evaluation framework • Matrix that associates evaluation questions with methods • Elements (rows) • Evaluation questions • Indicators (how would we know?) • Data sources (from whom to gather evidence) • Methods (how evidence will be gathered • Basis for Comparison (how good is good enough?)

  15. Evaluation Framework

More Related